In order to add some datas for the frequently asked question: “what vorbis encoder should I use”, I've decided to run a listening test, comparing four different version of vorbis at 160...165 kbps:
- CVS (oggenc 2.3)
- GT3b2 tuning (associated to 1.01 reference code)
- aoTuV beta 2
- aoTuV beta 2+ QK tuning
The choice of settings is directly based on phong and Mac useful works:
http://www.hydrogenaudio.org/forums/index....ndpost&p=215978 (http://www.hydrogenaudio.org/forums/index.php?showtopic=21916&view=findpost&p=215978)
http://www.hydrogenaudio.org/forums/index....ndpost&p=216019 (http://www.hydrogenaudio.org/forums/index.php?showtopic=21916&view=findpost&p=216019)
I. PURPOSE OF THIS TEST
I focused the test on one problem: pre-echo (in other words, sharpness, or edge details). Conclusions are therefore limited to this unique problem. Why pre-echo only? In my opinion, and under -q 6 setting, aoTuV code (beta 2) outperforms CVS code with or without Garf Tuning on all problems, except one: pre-echo. Overall sound with aoTuV at mid/low settings is cleaner, with less noise, and not as fat and coarse than with CVS code. It's just my opinion, based on personnal listening test (with classical music samples).
Pre-echo performances of aoTuV are for me a total mystery. I didn't really test this encoder on that point (I'm more annoyed by coarse sounding of vorbis). I've read that aoTuV include some pre-echo tuning, so I suppose that aoTuV performs better (but how much?) than CVS. More important question is: are aoTuV performances comparable to the nice tunings of Garf?
Garf Tuning (GT3) is very impressing on pre-echo samples. The only problem with this tuning: it's based on CVS code, which suffers at -q5...-q5.99 from serious problems (described as hiss, tonality difference, noise, coarse sound, stereo imaging...).
If aoTuV compete with GT3b2, I suppose that we could conclude on its overall superiority. But if aoTuV has more smearing issues, the question of “recommanded encoder” would probably stay problematic.
In addition to this test, the hybrid encoder named aoTuV+QK. I perfectly know that QK code implementation on aoTuV is problematic, ruining sometimes the positive effect of aoTuV tuning. Nevertheless, I wonder if these negative effects couldn't be balanced by positive performances of sharpness. The test should give some elements of answers.
II. SAMPLES.
For this test, I used more than 20 samples. Some of them are well-known: castanets, castanets2, c44... But most of them have my library as origin. I used short one, in order to upload them. I must add that most of my samples are not so "pure" or precise than the three previous one: guitar, marimbas, harpsichord, drums... couldn't compete with castanets for sharpness. Therefore, these samples are a bit harder to ABX. But they are maybe more representative, I don't know....
In addition to sharp and ponctual attacks represented in samples like castanets or percussions, I've add some samples with micro-attacks (like fatboy, but this one isn't present in this test). Lossy encoders tend to encode this kind of signal with extra-noise, more or less annoying. Four samples corresponds to this signal: awe32 (well-known - electronic music), Hmong (traditional vietnamese instrument), Orion II (solo trombone, one of the most problematic "occidental" instrument for lossy encoders), and Pierres Réfléchies (electronic/concrete music). I could add the beginning of the "creaking [door]" sample.
Funny thing to note: bitrate with the "guimbard" vietnamese instrument is terribly high : 280 kbps with CVS encoder, and up to 461 kbps with GT3b2 at -q 5.00 (full track is 'only' 450 kbps).
III. RESULTS
(http://audiotests.free.fr/tests/2004/results_preecho.png)
IV. CONCLUSIONS
• without any doubt, vorbis CVS performs poorly on the pre-echo problem. It's a well-known problem, no need to insist.
• GT3b2 tunings are impressing, metamorphosing the original CVS code. It's especially true for "pure pre-echo samples" (like castanets), but progress are clearly audible on other samples too. On micro-attacks, the bitrate explode completely (up to +60% on Hmong.wav), but quality is always in consequence: very nice.
• aoTuV beta 2 performances on pre-echo are simply remarquable. But the overall notation must be analysed:
- on very sharp attacks, aoTuV suffers a lot from smearing, and is not really far from CVS original code. Pity...
- on not-too-sharp attacks, aoTuV performs very well. Pre-echo is very limited. Comparable but probably slightly inferior to GT3b2.But without the extra-brightness and the irritating noise audible with GT3b2, quality of both encoders is similar.
- on micro-attacks, aoTuv is superior to CVS, but GT3b2 is the uncontestable winner.
• aoTuV+QK. The “winner” of this pre-echo test (best overall notation). Quantum Knot modifications are very pertinent on these kind of samples.
- on very sharp attacks, performances are slightly better than GT3b2! and outperforms aoTuV reference code.
- but on micro-attacks, quality is inferior to the original aoTuV code (for my taste; results might differs from other people).
- on not-too-sharp attacks, quality is close to aoTuV (sometimes more; sometimes less...), and is better than GT3b2 which suffers from other problems.
• there's not only pre-echo in music. The sample “Die Schlacht.wav” is a cruel reminiscence of this fact. I expected pre-echo on attacks: the four encodings were free of this problem. But I was badly surprised by the sound of strings, severly wounded by CVS and GT3 tunings. The responsible is maybe the problematic lossy-stereo model of Vorbis, bad or untuned. aoTuV lowered the problem, which is nevertheless still audible... Additional tunings are therefore welcome :-)
Wow, you truly are a star - thanks Guru, very interesting reading!
Though it's closer to a "192 kbps" test, than a 160 one... Interesting none the less
http://www.hydrogenaudio.org/forums/index....ndpost&p=215978 (http://www.hydrogenaudio.org/forums/index.php?showtopic=21916&view=findpost&p=215978)
According to these statistics, -q 5.00 and -q5.50 are ~160 kbps for general music (full CD encodings). On short and problematic sample, higher bitrate are something common and expected.
It is always a pleasure to read these detailed and meticulous listening tests from guruboolez. My eyes were glued to the screen till I reached the last full stop.
On average, it seems aoTuV does a bit better than GT3b2 as the former top-scored in 10 of the 22 samples while GT3b2 only 6. Also, my suspicion is that perhaps the hiss, coarseness, and noisy nature of CVS/GT3b2 is more annoying than smearing of transients and pre-echo, hence, coupled with harashin's findings, I think it is time for aoTuV to be the recommended coder for all q's.
Can I have a show of hands on whether we should retire GT3b2?
I vote to retire GT3b2 and have aoTuV as the recommended encoder at q > 5
EDIT: Changed 20 to 22 samples
In order to add some datas for the frequently asked question: “what vorbis encoder should I use”, I've decided to run a listening test, comparing four different version of vorbis at 160...165 kbps:
- CVS (oggenc 2.3)
- GT3b2 tuning (associated to 1.01 reference code)
- aoTuV beta 2
- aoTuV beta 2+ QK tuning
Very interesting to see that aoTuV ties with GT3 even on those kind of samples. I'd like to do some tests which feature my(and other people's) pre-echo samples on this weekend.
According to these statistics, -q 5.00 and -q5.50 are ~160 kbps for general music (full CD encodings). On short and problematic sample, higher bitrate are something common and expected.
Indeed... Sorry for the useless comment
Can I have a show of hands on whether we should retire GT3b2?
I've been using aoTuV only, since Roberto's last listening test So you have my vote
Note that aoTuV is slightly inferior to GT3b2 on average, if we remove the sample named "Die schalcht": GT3b2 is loosing two points on a problem which have nothing to do with pre-echo.
Before removing GT3b2, take a look to the average notation of "pure pre-echo" file (i.e. file with very strong and sharp attacks):
- c44
- castanets
- castanets2
- cataclysmes
- clapping
- creaking
I don't have notes in mind, but IIRC it's something like 2.3 / 5 for aoTuV and 3.6 for GT3b2.
aoTuV+QK is maybe the best vorbis compromise of the moment: it's the sharpest encoder, and with a lot of correction of hiss, noise, etc...
I've upload all samples (except 41_30) in optimfrog format here:
ftp://ftp2.foobar2000.net/foobar/ (http://ftp://ftp2.foobar2000.net/foobar/)
Someone should test the archive. They are maybe corrupted.
I can't upload them on HA sever now
Someone should test the archive. They are maybe corrupted.
I can't upload them on HA sever now
Both archives are fine here. (12+9 *.ofr files)
Edit: wrong number
Edit2: 41_30sec which isn't included in the archive, is available at ff123's site (http://ff123.net/samples.html).
Note that aoTuV is slightly inferior to GT3b2 on average, if we remove the sample named "Die schalcht": GT3b2 is loosing two points on a problem which have nothing to do with pre-echo.
Before removing GT3b2, take a look to the average notation of "pure pre-echo" file (i.e. file with very strong and sharp attacks):
- c44
- castanets
- castanets2
- cataclysmes
- clapping
- creaking
I don't have notes in mind, but IIRC it's something like 2.3 / 5 for aoTuV and 3.6 for GT3b2.
aoTuV+QK is maybe the best vorbis compromise of the moment: it's the sharpest encoder, and with a lot of correction of hiss, noise, etc...
Yeah, it all comes down to a compromise but do you think that pre-echo is a more 'forgiveable' problem for a lossy perceptual coder than hiss and noise? That is why I think aoTuV is probably more 'well-rounded', if we place more emphasis on its hiss and noise suppression rather than its higher pre-echo.
I think it is disappointing that aoTuV+QK regressed from aoTuV, though it shouldn't be unexpected, considering how I just slapped the two tunings together without any testing. But it certainly means that more effort is needed to tune it. I might have an idea on how to improve it on the micro-attack samples.
Yeah, it all comes down to a compromise but do you think that pre-echo is a more 'forgiveable' problem for a lossy perceptual coder than hiss and noise?
For me, without an hesitation. I have more violins than castanets in my CD library
Neverteless, some people listening a lot of sharp electronic music and sensitive to pre-echo, GT3b2 is maybe preferable. I don't know...
In my opinion, aoTuV is more complete to GT3b2; but aoTuV+QK seems to be more equilibrated: good trade-off between noise performance and sharpness.
EDIT: I've forgot to post the log files of the test:
here (http://audiotests.free.fr/tests/2004/RESULTS_FILES.zip)
I used ff123 abc/HR 1.1 beta for this test.
I am one of those people for whom the noise/hiss (at the level present in gt3b2) is less offensive than preecho. However, I will agree with the choice to make aotuv the recommended encoder because:
- I think I'm unusually sensetive to preecho and transient smearing compared to other artifacts
- The noise problem seems to be more common than preecho on average
- The advantages of having one recommended encoder are worth it
- I'm "in the know" so if I want to use gt3b2 on my Aphex Twin albums, I will know to do that
- aotuv has been reported to fix the sometimes very annoying stereo issues with vorbis below -q 6
I guess it's also too early to rule out your experimental aotuv+QK as it did well here. Even though it regresses compared to aotuv it's still definately better than stock and may represent a decent comprimise between "fixing noise" and "fixing preecho".
One thing that's definately shown by these test is that vorbis is still in need of a quite a bit of tuning based on the number of samples that have problems even at high -q levels (compared to e.g. mpc.)
Hello.
Do the answer please though you are silly question.
Can any one explain what pre-echo is?
Thanks for any help or advice.
Do the answer please though you are silly question.
Can any one explain what pre-echo is?
Just look into the page (http://ff123.net/preecho.html) from ff123 web site.
Anyway....thanks Guru for these great tests
I suggest you retest aoTuV b2 at -q6 (to compare with GT3 b2 -q5). There is certainly enough "bitrate room" and personally I find aoTuV b2 to perform a bit better on pre-echo at 6.00+ than 5.00-5.99. To my ears, it's transparent on castanets at -q6. It's has that smoothed-over sound at 5.99 and under.
EDIT: I understand why aoTuV b2 was used at -q5.5 (instead of -q6). My hypothesis, however, is that aoTuV -q6 will be better than GT3 -q5.99.
Thanks for replies
About aoTuV+QK regression (compared to aoTuV): I wonder how often and in what precise conditions it happens.
I graphically compared a solo violin piece encoded with aoTuV and aoTuV+QK. Why violin? Violin is a tonal instrument. QuantumKnot code shouldn't therefore modify the file. A graphical comparison confirms that, and it *shows* that *objective* difference is sub-existant:
• whole file: http://membres.lycos.fr/guruboolez/AUDIO/v..._difference.png (http://membres.lycos.fr/guruboolez/AUDIO/violin_difference.png)
• 10 seconds zoom: http://membres.lycos.fr/guruboolez/AUDIO/v...rence_10sec.png (http://membres.lycos.fr/guruboolez/AUDIO/violin_difference_10sec.png)
Filesize is exactly the same (2kb only difference). Few samples are modified. It's really nothing, and can't modifiy the noise performances of aoTuV code. Good thing
For comparison, here's a difference between aoTuV+QK and GT3b2:
http://membres.lycos.fr/guruboolez/AUDIO/v...GT3_aoTuVQK.png (http://membres.lycos.fr/guruboolez/AUDIO/violin_difference_GT3_aoTuVQK.png)
(P.S. all encodings were done at the same setting: -q 5,00)
(P.S.2 Before someone complain about TOS infringing, I've also tried to ABX the files: no difference between aoTuV/aoTuVQK and obvious difference in favor of aoTuV compared to GT3b2).
My hypothesis, however, is that aoTuV -q6 will be better than GT3 -q5.99.
Testing aoTuV at -q 6,22 and GT3b2 at -q 6,00 is more interesting in my opinion than testing GT3b2 at -q 5,99 against anything. I don't want to offend any developer, but performances of CVS/GT3 below -q 6.00 are disappointing in regard to bitrate. Instruments generally sound fat, coarse (it's limited compared to -q 4.00, but it's inacceptable for encodings close to 200 kbps).
Aotuv anyday. The CVS/GT3 <Q6 noise issue for the high bitrate is bad and can be audiable in normal listening. Q6 or higher differences between GT3 & AoTuv will probably only exist in abx situations - I may be wrong though.
At least we will have *one* good general purpose encoder in aotuv. It will help boost confidence in vorbis i think.
A dumb question, why aren't GT3, AoTuv and QK modifications available in the Xiph CVS (SVN) ? Like an alternative choice on command-line.
AFAIK LAME has some different command-line choices for different algos.
QK, I would be inclined to go with the increasing number of tests showing either of the AoTuV tunings to be superior to GT3 at q5+ It is time for one Vorbis to rule them all, or something like that..
Aotuv anyday. The CVS/GT3 <Q6 noise issue for the high bitrate is bad and can be audiable in normal listening. Q6 or higher differences between GT3 & AoTuv will probably only exist in abx situations - I may be wrong though.
One preecho sample I have is detectable (but not particularly serious) in what I would consider "normal listening" at -q 6. It's improved (but not quite "fixed") by gt3b2 compared to aotuv and 1.0.1. I am going to test the aotuv+qk version this weekend with this sample and some other preecho samples; I think it may have been counted out too soon!
Thanks for replies
About aoTuV+QK regression (compared to aoTuV): I wonder how often and in what precise conditions it happens.
I graphically compared a solo violin piece encoded with aoTuV and aoTuV+QK. Why violin? Violin is a tonal instrument. QuantumKnot code shouldn't therefore modify the file. A graphical comparison confirms that, and it *shows* that *objective* difference is sub-existant:
• whole file: http://membres.lycos.fr/guruboolez/AUDIO/v..._difference.png (http://membres.lycos.fr/guruboolez/AUDIO/violin_difference.png)
• 10 seconds zoom: http://membres.lycos.fr/guruboolez/AUDIO/v...rence_10sec.png (http://membres.lycos.fr/guruboolez/AUDIO/violin_difference_10sec.png)
Filesize is exactly the same (2kb only difference). Few samples are modified. It's really nothing, and can't modifiy the noise performances of aoTuV code. Good thing
For comparison, here's a difference between aoTuV+QK and GT3b2:
http://membres.lycos.fr/guruboolez/AUDIO/v...GT3_aoTuVQK.png (http://membres.lycos.fr/guruboolez/AUDIO/violin_difference_GT3_aoTuVQK.png)
(P.S. all encodings were done at the same setting: -q 5,00)
(P.S.2 Before someone complain about TOS infringing, I've also tried to ABX the files: no difference between aoTuV/aoTuVQK and obvious difference in favor of aoTuV compared to GT3b2).
That narrows the regression down a bit. The QK component affects only short blocks, hence this agrees with the findings from your violin sample, which are mostly long blocks. The regression must be occurring in regions of transient attacks, probably boosting the HF hiss caused by point stereo.
I'm back, with the same test, but at higher setting: -q 6.00...6.50. Samples are the same. I was nevertheless short in time, and I didn't ABXed the last files. For these files, the difference in notation may be imprecise (it means that small differences in notation and hierarchy are maybe unjustifed).
For those wondering about setting used for CVS encoder (-q 6,50): I don't have any internet access at home, and I hadn't the correspondant values found by phong when I started the test. I've at first encoded all files with CVS -q 6,22 like aoTuV, but I feared that bitrate was too low. Therefore, I've decided to round the setting to a nice 6,50. It's probably a bit too high, but quality shouldn't really change between the setting I used and the ideal one (6,36 according to phong bitrate table).
RESULTS
(http://audiotests.free.fr/tests/2004/results_preecho_q6.png)
Log files are here (few or even no comments):
here (http://audiotests.free.fr/tests/2004/RESULTS_FILES_Q6.zip)
CONCLUSIONS
• CVS encoder have serious troubles with transients, even at -q 6,50 (210 kbps nominal, which is a pretty high bitrate for a lossy encoder). Something like brightness is also audible with some files, but it's not really annoying, and very far from thickness/coarseness heard previously at -q 5,50, which considerably lowered the notation. In other words, quality progress a lot between -q 5,50 and -q 6,50; pre-echo is in my opinion the biggest problem (but not the only one) of CVS encoders at -q 6...9 settings.
• In this conditions, a CVS encoder tweaked for pre-echo should be impressive. And it's the case for GT3b2, which progressed a lot between the two tests (-q 5.00 then -q 6.00). Incidentally, 30% of the samples were transparent on my test (I could probably find more differences with insane concentration). Interesting to note: the sample "Die Schlacht", symptomatic of the coarseness of vorbis, was here fully transparent on violins with both CVS and GT3b2. Other interesting point: micro-attacks (creaking, Hmong, Pierre Réfléchies, Orion II) are still better with GT3b2 than with any other vorbis encoder, though extra-noise is still perceptible.
• aoTuV's performances are now between CVS and GT3b2. It's always better than CVS (except one case, but it was on quick test, without ABX: notation is a bit imprecise, and hierarchy might be wrong). But it's rarely better or simply eaqual than GT3b2. On very sharp attacks (castanets...), aoTuV performances are a bit disapointing. We could expect more from a modern lossy encoders. On moderately sharp attacks, pre-echo is limited, not really annoying.
We could also note that the progress between the notation on the two tests is small. aoTuV -q 6,22 is just slightly better than -q 5,50, whereas progress with CVS and GT3b2 is very impressive. It's not a real problem, quite the reverse: it proves that aoTuV quality is more linear than CVS/GT3, without huge frontier between - q 5,99 and -q 6,00.
Last thing: when brightness is audible with CVS/GT3, aoTuV hasn't problem, or lowers it (cf. Atem-Lied for a good exemple).
• In conclusion, aoTuV needs more tuning in order to be •fully• recommanded over GT3b2. A temporary solution might be something like an aoTuV+GT3 encoder. Ready for another round?
very interesting.
by the way, guruboolez, do you think about "aoTuV beta 1a with GT3's pre-echo tune (http://www.hydrogenaudio.org/show.php/showtopic/19925)" at 160kbps or -q 5? i want to know your test result very much.
Guruboolez, thank you very much! I believe you make a valuable contribution to Vorbis development. But may I advise you to use LAME APS instead of official Vorbis encoder since the last one is the real outsider (if you are going to do any tests again). I believe it will collect much more audience and results will be very very welcome by HA community. IMHO. And thanks a lot again.
madoka@ex-sounds> I'm not really interested to test an experimental/outdated encoder. We don't know anything about aoTuV beta 1 performances at any level (pre-echo, noise, etc...). A patched version of aoTuV beta2 is preferable in my opinion, because we approximately know how performs this version on many situation.
de Mon> I don't want to mix multiformat and multicodec in the same test. Especially when the sample are limited to one specific problem. Lame APS will probably lose against vorbis on pre-echo, even against CVS encoder. Challenge is interesting, but with various problem samples.
I used CVS in order to have an idea about performances of aoTuV compared to a reference code. It's really important, to prove (or conclude) that a "fork" encoder is better than the reference one.
de Mon> I don't want to mix multiformat and multicodec in the same test. Especially when the sample are limited to one specific problem. Lame APS will probably lose against vorbis on pre-echo, even against CVS encoder. Challenge is interesting, but with various problem samples.
I used CVS in order to have an idea about performances of aoTuV compared to a reference code. It's really important, to prove (or conclude) that a "fork" encoder is better than the reference one.
Thanks for reply. Anyway it was interesting to know your opinion.
Thanks for the test, Guruboolez.
Would it be possible to input all ranks in a program that would draw a graph with error bars ? Reading and comparing all ranks to check for consistency or contradictions is a pain.
I've done a quick merge of aoTuV beta 2 and GT3b2:
http://www.rarewares.org/quantumknot/oggencagt.exe (http://www.rarewares.org/quantumknot/oggencagt.exe)
Use for testing only. There might be some values from GT3b2 which I haven't ported over but the most important ones, such as pre-echo triggers and impulse block tunings are included.
Merge sounds pretty good. Well at least I can't tell it apart from my gt3b2 files which I couldn't tell apart from the CD. Can't say I tried a really hard sample with it though and I only tried a few files.
QuantumKnot> does this version include Garf tunings (i.e. from -q 5,00 to -q 10,00) or are your own tunings (-q 2 to -q 5) also build-in?
I'll test this encoder, but it's difficult for me to do it before wednesday.
QuantumKnot> does this version include Garf tunings (i.e. from -q 5,00 to -q 10,00) or are your own tunings (-q 2 to -q 5) also build-in?
I'll test this encoder, but it's difficult for me to do it before wednesday.
This is just aoTuV along with Garf's stuff (-q 5 to 10). There are no tunings of mine (-q 2 to 5) but if you want, I can add them in when I'm free...a sort of megamix
It would be nice. Imagine that aoTuV+GT3 will appear to be the best solution on pre-echo sample at -q5...10 on a very soon test (TEST NO.2, to be done). aoTuV+QK is already the best (according to my limited test: TEST NO.1, finished) at lower bitrate. I suppose that people will request this megamix, in order to have a federative vorbis encoder. Then, a new listening test will be needed, in order to be sure that merging three different tunings (aoTuV; GT3; QK32) don't break anything (TEST NO.3 - to be done).
Releasing the megamix vorbis encoder is a good opportunity to spare time and motivation, because TEST NO.2 won't be needed anymore
(sorry for childish explanation)
It would be interesting to find out how all these new Vorbis variants compare to the "benchmark" of mppenc 1.14 --standard --xlevel. Transient handling/Pre-echo mitigation has always been a major strength of musepack so it could be helpful to see how far Vorbis has gotten in comparison.
I'm also very interested by this comparison, but I'd like to test a good AAC encoder too (a gapless one, iTunes encoder is exclude). I'm therefore waiting for a major update of Nero AAC before starting this comparison.
vorbis >200kbps seems useless to me, so perhaps (if others agree, obviously...) <=q6 should be the target comparisons.
at any rate, it's good to see so much interest in free formats, and as always, you guys (guru/harashin/QK/etc.) probably keep vorbis afloat in the midst of other progressing/superior formats
just an idea... but has modest tuning been tested much above the q4(...) setting of the ogg vorbis contendor tests? ...it seems premature to leave it out.
thanx!
later
I've done a quick merge of aoTuV beta 2 and GT3b2:
Would it be possible to get sources for this? I'd like to make a linux version.
It would be nice. Imagine that aoTuV+GT3 will appear to be the best solution on pre-echo sample at -q5...10 on a very soon test (TEST NO.2, to be done). aoTuV+QK is already the best (according to my limited test: TEST NO.1, finished) at lower bitrate. I suppose that people will request this megamix, in order to have a federative vorbis encoder. Then, a new listening test will be needed, in order to be sure that merging three different tunings (aoTuV; GT3; QK32) don't break anything (TEST NO.3 - to be done).
Releasing the megamix vorbis encoder is a good opportunity to spare time and motivation, because TEST NO.2 won't be needed anymore
(sorry for childish explanation)
Sounds fair enough. I'll do the megamix merge. q 5 is common between Garf's tunings and mine. Which one is preferred?
It would be interesting to find out how all these new Vorbis variants compare to the "benchmark" of mppenc 1.14 --standard --xlevel. Transient handling/Pre-echo mitigation has always been a major strength of musepack so it could be helpful to see how far Vorbis has gotten in comparison.
In a very general sense, throw enough bits at it, and it will probably sound sharper. That seems to be what pre-echo improved Vorbis encoders seem to do, hence the wild bitrate increases whenever there are sharp attacks. q 5 is one place where I've tried to get the sharpness of GT3b2 but with lesser bits.
Musepack, being a subband coder, has better time resolution, so it generally will need to throw less bits at transients than minimalistic transform coders, like Vorbis, to give the same quality. But since these tests are performed at q values determined to give a target bitrate on real-life music (rather than avg bitrate on killer samples), then it would be interesting to compare, yes.
q 5 is common between Garf's tunings and mine. Which one is preferred?
no idea. Wasn't GT3b2 recommanded at -q 5 up to a recent date? So maybe this one
q 5 is common between Garf's tunings and mine. Which one is preferred?
no idea. Wasn't GT3b2 recommanded at -q 5 up to a recent date? So maybe this one
No problem.
I was busy and my condition wasn't good enough to perform such tests last weekend. I even failed to ABX some encoders on castanets.
I found aoTuVb2 + GT3 is my preferable Vorbis encoder at -q5 on this sort of samples. It's worth to mention that some encoders are distinguishable for other reasons than pre-echo. (Mostly HF boost)
(http://cyberquebec.ca/harashin/pre-echo_test.png)
pre-echo_results.zip (http://cyberquebec.ca/harashin/pre-echo_results.zip)
Edit: I can't upload test samples at the upload forum for some reason.
Edit2: Result for trust is replaced with latest one.
Thanks Harashin. Wow, what happened with aoTuV+QK on trust?
Wow, what happened with aoTuV+QK on trust?
Somehow I felt it was brighter than others. I'll test trust sample again tomorrow.
Wow, what happened with aoTuV+QK on trust?
Somehow I felt it was brighter than others. I'll test trust sample again tomorrow.
Thanks. If it is true that it appeared brighter, compared with the aoTuV+GT3b2, then it pinpoints a particular set of values that Garf changed which I didn't touch and that may prove useful. Awesome
QuantumKnot,
I would like to join Phong and ask for the source. In fact every version you create, please upload the source too. I am on linux, but linux compile will not do it. As a matter of principle I always need the source.
PS: After painful deliberation, I moved from Lame to Vorbis and I am just about to kick off transcode my entire already FLAC-ed archive to Vorbis. So I follow the developments of the last few months with a lot of interest. You, Garf, Aoyumi, Guruboolez, Harashin and Phong rock!!! You guys are the heros of Vorbis. I hope that Monty does take notice. OK maybe the man just wait until the new HA recommended Vorbis emerges. But then I will really expect him to incorporate these changes.
Many thanks for your hard work.
OK, here is the binary for Vorbis Megamix (aoTuV beta 2 + QKTune beta 3.2 + GTune 3 beta 2). From q -2 to 1 is aoTuV only. From q 2 to 4 is QKTune + aoTuV. And from q 5 to 10 is GT3b2 + aoTuV. I haven't tested this binary much (other than look at average bitrates) so use it for testing only and report any problems here. Bitrate versus quality over the whole range would be a very interesting graph.
Megamix Binary:
http://www.rarewares.org/quantumknot/oggenc-megamix.exe (http://www.rarewares.org/quantumknot/oggenc-megamix.exe)
Megamix DLLs:
http://www.rarewares.org/quantumknot/vorbis-megamix-dlls.zip (http://www.rarewares.org/quantumknot/vorbis-megamix-dlls.zip)
Vendor tag: Vorbis Megamix: aoTuV beta 2 + QKTune beta 3.2 + GTune 3 beta 2
For those who requested the source files, they are below:
http://www.rarewares.org/quantumknot/vorbis-megamix-src.zip (http://www.rarewares.org/quantumknot/vorbis-megamix-src.zip)
http://www.rarewares.org/quantumknot/vorbi...tuvqk32-src.zip (http://www.rarewares.org/quantumknot/vorbis-aotuvqk32-src.zip)
http://www.rarewares.org/quantumknot/vorbi...uvgt3b2-src.zip (http://www.rarewares.org/quantumknot/vorbis-aotuvgt3b2-src.zip)
If you are familiar with the GT3b2 source code and have some spare time on your hands, do a cross-check between the megamix and aotuvgt3b2 merge sources to see if I have left anything out. I've done my best in limited time to include the most important tunings (short block, pre-echo trigger, etc), but there may be something I missed. Thanks.
EDIT: Updated binary and source so it has the proper vendor tag
You are too awesome QK
I'm playing around with it a little now. I'll put it up to hard samples later. Sounds pretty good on some regular music though
You are too awesome QK
I'm playing around with it a little now. I'll put it up to hard samples later. Sounds pretty good on some regular music though
Some ABC/HR tests would be nice
I've just uploaded my test samples at my hosting space. Other samples are probably found with using search function of the forum.
Diner_reprise.flac (http://cyberquebec.ca/harashin/Diner_reprise.flac)
Khan.flac (http://cyberquebec.ca/harashin/Khan.flac)
Shostakovich_sym14.flac (http://cyberquebec.ca/harashin/Shostakovich_sym14.flac)
You, Garf, Aoyumi, Guruboolez, Harashin and Phong rock!!
Ack, how did I get in that list? I've only managed to finish one ABC/hr test.
Anyhow, I've compiled QK's new versions and I've started my computer grinding away to complete the bitrate vs. quality table. To save time (so I don't have to encode too many combinations), I'm going to stick with the same set of songs as before. I'll post an update as soon as it's done with the most interesting bitrate range (4 <= q <= 7 or so).
Since a couple of people have indicated my average bitrates are lower than what they're seeing (in particular, that gt3b2 bitrates are not high enough relative to other versions), I'm going to add a couple songs that really bloat with gt3b2. That'll take a while though, so I'm gonna save it for later.
Megamix Binary:
http://www.rarewares.org/quantumknot/oggenc-megamix.exe (http://www.rarewares.org/quantumknot/oggenc-megamix.exe)
can't access it
edit: works now, maybe i should not scream immediately
Megamix Binary:
http://www.rarewares.org/quantumknot/oggenc-megamix.exe (http://www.rarewares.org/quantumknot/oggenc-megamix.exe)
can't access it
No problems here. Rarewares often has connection problems so try a few times
This result is quite different from my result which had been done yasterday. I found notable artifact in background and this is rather annoying than pre-echo. Therefore, although this is actually an easy-to-ABX sample, I won't do to use this sample for pre-echo only test anymore.
Edit: Link for useless result is deleted to avoid any confusing. Read below post.
Edit2:I believe this is a reasonable result.
Trust_Vorbis_test (http://cyberquebec.ca/harashin/Trust_Vorbis_test.htm)
Thanks. hmm....can someone do a bit-by-bit comparison between aoTuV+GT3b2 and megamix at q 5 on a sample? I didn't touch anything else so they should be almost identical.
Thanks. hmm....can someone do a bit-by-bit comparison between aoTuV+GT3b2 and megamix at q 5 on a sample? I didn't touch anything else so they should be almost identical.
I can confirm that. Oh, my result for that sample is completely useless.
You, Garf, Aoyumi, Guruboolez, Harashin and Phong rock!!! You guys are the heros of Vorbis.
Don't forget to include nyaochi as well. He is also a third-party Vorbis developer too.
Here's the table as promised. I'll add to it when I get results for other -q ranges.
Average bitrates
-q xiph.org aotuv a+gt3b2 a+qk32 gt3b2 megamix qk32
4.00 120.19 121.45 121.45 132.05 120.19 132.05 124.10
4.50 131.32 132.79 140.29 143.91 138.47 143.94 136.46
4.99 142.58 144.24 155.56 154.84 152.78 155.62 148.25
5.00 151.13 150.45 161.67 160.89 161.02 161.67 157.55
5.50 161.24 162.27 171.43 168.44 170.73 171.43 164.14
5.99 172.02 174.64 181.41 174.77 181.04 181.41 168.43
6.00 180.15 182.24 188.42 182.25 188.26 188.42 179.56
6.50 191.42 195.83 199.63 195.83 200.20 199.63 191.10
6.99 203.14 209.59 211.36 209.59 212.52 211.36 203.13
7.00 205.65 214.89 218.31 214.89 218.51 218.31 205.65
Equivalent -q levels
-q xiph.org aotuv a+gt3b2 a+qk32 gt3b2 megamix qk32
4.00 4.53 4.47 4.28 4.00 4.32 4.00 4.32
4.50 4.80 4.73 4.43 4.24 4.49 4.26 4.57
5.00 5.52 5.47 5.00 5.05 5.03 5.00 5.31
5.50 5.77 5.69 5.29 5.40 5.33 5.29 5.84
6.00 6.37 6.23 6.00 6.23 6.01 6.00 6.38
6.50 6.70 6.51 6.36 6.51 6.34 6.36 6.71
They look quite consistent, esp. how megamix at q 4 and q 5 has the same average bitrate as aotuv+qk and aotuv+gt3b2 respectively.
Updated!vorbis_bitrates.sxc (http://www.phong.org/audio/vorbis_bitrates.sxc) OpenOffice.org
vorbis_bitrates.xls (http://www.phong.org/audio/vorbis_bitrates.xls) MS Excel
Here's the complete table.
Average bitrates bitrate vs. xiph.org
aotuv aotuv aotuv aotuv
-q xiph.org aotuv qk32 gt3b2 +qk32 +gt3b2 megamix xiph.org aotuv qk32 gt3b2 +qk32 +gt3b2 megamix
-1.00 47.3 48.1 43.6 47.3 48.1 48.1 48.1 0.0% 1.6% -7.8% 0.0% 1.6% 1.6% 1.6%
-0.50 53.3 53.3 49.5 53.3 53.3 53.3 53.3 0.0% 0.0% -7.2% 0.0% 0.0% 0.0% 0.0%
-0.01 60.3 59.2 56.3 60.3 59.2 59.2 59.2 0.0% -1.8% -6.6% 0.0% -1.8% -1.8% -1.8%
0.00 62.9 62.5 58.9 62.9 62.5 62.5 62.5 0.0% -0.6% -6.3% 0.0% -0.6% -0.6% -0.6%
0.50 71.1 70.1 66.5 71.1 70.1 70.1 70.1 0.0% -1.4% -6.5% 0.0% -1.4% -1.4% -1.4%
0.99 78.9 77.9 73.7 78.9 77.9 77.9 77.9 0.0% -1.3% -6.6% 0.0% -1.3% -1.3% -1.3%
1.00 78.3 77.2 74.0 78.3 77.2 77.2 77.2 0.0% -1.4% -5.6% 0.0% -1.4% -1.4% -1.4%
1.50 81.6 82.3 82.6 81.6 88.7 82.3 88.7 0.0% 0.8% 1.2% 0.0% 8.6% 0.8% 8.6%
1.99 84.8 87.5 88.2 84.9 97.2 87.5 97.2 0.0% 3.1% 4.0% 0.0% 14.6% 3.1% 14.6%
2.00 89.1 92.6 92.2 89.1 102.5 92.6 102.5 0.0% 3.9% 3.5% 0.0% 15.0% 3.9% 15.0%
2.50 95.0 97.5 98.6 95.0 108.2 97.5 108.2 0.0% 2.6% 3.7% 0.0% 13.9% 2.6% 13.9%
2.99 100.7 101.8 104.5 100.7 112.9 101.8 112.9 0.0% 1.2% 3.8% 0.0% 12.1% 1.2% 12.1%
3.00 108.9 109.9 115.3 108.9 122.9 109.9 122.9 0.0% 0.9% 5.8% 0.0% 12.9% 0.9% 12.9%
3.50 115.5 117.5 122.5 115.5 131.6 117.5 131.6 0.0% 1.7% 6.0% 0.0% 13.9% 1.7% 13.9%
3.99 121.8 124.5 129.2 121.8 139.4 124.5 139.4 0.0% 2.2% 6.1% 0.0% 14.4% 2.2% 14.4%
4.00 121.8 122.8 129.3 121.8 137.7 122.8 137.7 0.0% 0.8% 6.1% 0.0% 13.1% 0.8% 13.1%
4.50 133.3 134.6 142.5 143.7 150.3 145.5 150.4 0.0% 1.0% 6.9% 7.8% 12.8% 9.1% 12.9%
4.99 144.8 146.4 154.6 159.3 161.4 162.5 162.5 0.0% 1.1% 6.8% 10.0% 11.4% 12.2% 12.2%
5.00 153.9 153.0 164.2 167.8 167.7 168.9 168.9 0.0% -0.5% 6.7% 9.1% 9.0% 9.8% 9.8%
5.50 164.3 166.2 169.9 177.7 174.9 179.1 179.1 0.0% 1.2% 3.4% 8.2% 6.5% 9.0% 9.0%
5.99 175.4 180.1 171.9 188.2 180.3 189.5 189.5 0.0% 2.7% -2.0% 7.3% 2.8% 8.0% 8.0%
6.00 183.5 187.5 182.9 195.0 187.5 195.9 195.9 0.0% 2.2% -0.3% 6.3% 2.2% 6.8% 6.8%
6.50 195.2 202.2 194.9 207.2 202.2 207.4 207.4 0.0% 3.6% -0.2% 6.2% 3.6% 6.2% 6.2%
6.99 207.4 216.9 207.4 219.9 216.9 219.3 219.3 0.0% 4.6% 0.0% 6.0% 4.6% 5.7% 5.7%
7.00 210.0 222.6 210.0 226.3 222.6 226.7 226.7 0.0% 6.0% 0.0% 7.8% 6.0% 8.0% 8.0%
7.50 226.2 240.3 226.2 240.4 240.3 245.2 245.2 0.0% 6.3% 0.0% 6.3% 6.3% 8.4% 8.4%
7.99 243.0 260.1 243.0 255.4 260.1 265.9 265.9 0.0% 7.0% 0.0% 5.1% 7.0% 9.4% 9.4%
8.00 242.0 258.8 242.0 253.9 258.8 264.4 264.4 0.0% 6.9% 0.0% 4.9% 6.9% 9.2% 9.2%
8.50 273.2 288.0 273.2 284.3 288.0 294.1 294.1 0.0% 5.4% 0.0% 4.1% 5.4% 7.6% 7.6%
8.99 306.9 318.5 306.9 317.2 318.5 325.2 325.2 0.0% 3.8% 0.0% 3.4% 3.8% 6.0% 6.0%
9.00 313.0 327.2 313.0 325.4 327.2 334.7 334.7 0.0% 4.5% 0.0% 4.0% 4.5% 7.0% 7.0%
9.50 400.2 412.5 473.8 495.6 412.5 440.5 440.5 0.0% 3.1% 18.4% 23.8% 3.1% 10.1% 10.1%
9.99 444.8 461.5 513.3 549.8 461.5 510.9 510.9 0.0% 3.8% 15.4% 23.6% 3.8% 14.9% 14.9%
10.00 445.3 462.2 513.9 550.6 462.2 511.9 511.9 0.0% 3.8% 15.4% 23.6% 3.8% 15.0% 15.0%
Equivalent -q levels (all) Equivalent -q levels (excluding outliers)
aotuv aotuv aotuv aotuv
-q xiph.org aotuv qk32 gt3b2 +qk32 +gt3b2 megamix xiph.org aotuv qk32 gt3b2 +qk32 +gt3b2 megamix
-1.00 -0.94 -1.00 -0.62 -0.94 -1.00 -1.00 -1.00 -0.94 -1.00 -0.62 -0.94 -1.00 -1.00 -1.00
-0.50 -0.54 -0.55 -0.27 -0.54 -0.55 -0.55 -0.55 -0.54 -0.55 -0.27 -0.54 -0.55 -0.55 -0.55
0.00 0.00 0.02 0.26 0.00 0.02 0.02 0.02 0.00 0.02 0.26 0.00 0.02 0.02 0.02
0.50 0.42 0.49 0.73 0.42 0.49 0.49 0.49 0.42 0.49 0.73 0.42 0.49 0.49 0.49
1.00 1.00 1.11 1.25 1.00 1.05 1.11 1.05 1.00 1.11 1.25 1.00 1.05 1.11 1.05
1.50 1.86 1.66 1.62 1.86 1.23 1.66 1.23 1.86 1.66 1.62 1.86 1.23 1.66 1.23
2.00 3.13 3.02 2.81 3.13 2.00 3.02 2.00 2.30 2.00 2.03 2.30 2.00
2.50 2.93 2.78 2.62 2.93 1.64 2.78 1.64 2.65 2.41 2.35 2.65 2.41
3.00 4.06 3.86 3.53 4.06 3.00 3.85 3.00 3.48 3.35 3.00 3.48 3.35
3.50 3.98 3.79 3.44 3.98 2.87 3.79 2.87 3.67 3.51 3.15 3.67 3.51
4.00 4.69 4.63 4.32 4.36 4.00 4.33 4.00 4.69 4.63 4.32 4.36 4.00 4.33 4.00
4.50 4.91 4.84 4.52 4.48 4.17 4.43 4.20 4.91 4.84 4.52 4.48 4.17 4.43 4.20
5.00 5.72 5.60 5.41 5.05 5.08 5.00 5.00 5.72 5.60 5.41 5.05 5.08 5.00 5.00
5.50 5.89 5.74 6.27 5.28 5.33 5.21 5.21 5.91 5.76 5.30 5.38 5.24 5.24
6.00 6.53 6.29 6.54 6.04 6.29 6.00 6.00 6.53 6.29 6.54 6.04 6.29 6.00 6.00
6.50 6.79 6.51 6.79 6.31 6.51 6.29 6.29 6.79 6.51 6.79 6.31 6.51 6.29 6.29
7.00 7.52 7.12 7.52 7.01 7.12 7.00 7.00 7.52 7.12 7.52 7.01 7.12 7.00 7.00
7.50 7.83 7.43 7.83 7.41 7.43 7.32 7.32 7.83 7.43 7.83 7.41 7.43 7.32 7.32
8.00 8.36 8.10 8.36 8.17 8.10 8.00 8.00 8.36 8.10 8.36 8.17 8.10 8.00 8.00
8.50 8.67 8.45 8.67 8.51 8.45 8.36 8.36 8.67 8.45 8.67 8.51 8.45 8.36 8.36
9.00 9.12 9.04 9.07 9.03 9.04 9.00 9.00 9.12 9.04 9.07 9.03 9.04 9.00 9.00
9.50 9.93 9.77 9.07 8.99 9.77 9.49 9.49 9.73 9.59 9.59 9.37 9.37
(http://www.phong.org/audio/vorbis_graph.png)
No problems here. Rarewares often has connection problems so try a few times
We do?
No problems here. Rarewares often has connection problems so try a few times
We do?
Very rarely so it's nothing to be alarmed about (I guess it's not just the wares that a rare ) I think there were one or two times when I had troubles connecting to it the first time but usually it worked if I tried connecting to it again. Probably more to do with my ISP.
EDIT: I noticed I used the word 'often' in my original post. I actually meant 'sometimes' .
[span style='font-size:14pt;line-height:100%']guruboolez can't access to HA, and asked to post the following formated text[/span]
SAMPLES
Same as before, with for new samples (three posted by harashin:
Diner_reprise,
Khan, Shostakovitch_sym14[/I] - another one, public:
trust). I have just change the glockenspiel one (
Zauberflöte ->
Zauberflöte (2)).
There are now 26 samples, and three categories: sharp attacks - micro-attacks - and the rest: attacks mixed with instruments.
CHALLENGERS
I removed CVS encoder in order to spare time and motivation. With 26 samples and three challengers, I have enough material to finish completely tired and with a big headache.
GT3b2 is tested (-q 5,00),
aoTuV beta 2 original code (-q 5,50) with
aoTuV MEGAMIX (-q 5,00) in regard.
RESULTS
(http://www.foobar2000.net/divers/tests/2004.06/results_megamix_q5.png)
CONCLUSIONS
• on sharp attacks, aoTuV b2 is the looser and suffers from the lack of power of the native pre-echo tuning. GT3b2 and aoTuV MEGAMIX are tied. On some samples, GT3b2 is better; on some other, it is slightly worse than aoTuV MEGAMIX. For this kind of sample (rare in my opinion),
GT3b2 and aoTuV MEGAMIX are both recommanded.
• on micro attacks,
GT3b2 is the clear winner. The MEGAMIX tunings doesn't reproduce the benefits of GT3b2. It's very obvious to prove with the
creaking sample (see Appendix below). And in one case, MEGAMIX is worse than pure aoTuV (Orion II, confirmed with a brilliant ABX test: 18/18).
GT3b2 is the the very best vorbis .version for micro-attacks.• on 'normal' transients,
the winner is aoTuV MEGAMIX. Sharper than pure
aoTuV, and much cleaner than
GT3B2. GT3b2 suffers too much from the fat rendering of CVS legacy. aoTuV is good, and aoTuV MEGAMIX is just better =>
I recommand aoTuV MEGAMIX for most common transient situations.• I kept the sample named "Die Schlacht". I can't differenciate the encoding from the original on transients, but on additional noise affecting ~tonal~ sound. GT3b2 suffered a lot. The MEGAMIX addition doesn't affect the nice quality of aoTuV reference code. In other word,
aoTuV MEGAMIX is apparently recommandable with non-transient samplesAPPENDIX.
The noisy artifact audible with the
creaking sample is clearly visible on a graphical representation:
http://www.foobar2000.net/divers/tests/200...iff_ref_GT3.gif (http://www.foobar2000.net/divers/tests/2004.06/diff_ref_GT3.gif)
http://www.foobar2000.net/divers/tests/200...ref_megamix.gif (http://www.foobar2000.net/divers/tests/2004.06/diff_ref_megamix.gif)
We could also see that benefits of GT3 tunings are not reproduced with the MEGAMIX one:
http://www.foobar2000.net/divers/tests/200...gamix_aoTuV.gif (http://www.foobar2000.net/divers/tests/2004.06/diff_megamix_aoTuV.gif)
APPENDIX (2)
I have also noticed two times the same "hollow sound" artifact,
audible with aoTuV based encoders, but not with GT3b2: with
clapping and with
shostakovich_sym14.
I've tried to make this artifact visible on Cool Edit. It's probably this hole visible in the 10 Khz band:
http://www.foobar2000.net/divers/tests/200...TuV_MEGAMIX.png (http://www.foobar2000.net/divers/tests/2004.06/clapping_aoTuV_MEGAMIX.png)
The GT3 file:
http://www.foobar2000.net/divers/tests/200...pping_GT3b2.png (http://www.foobar2000.net/divers/tests/2004.06/clapping_GT3b2.png)
[span style='font-size:14pt;line-height:100%']guruboolez can't access to HA, and asked to post this second formated text[/span]
I was too bored to complete the -q 6 test. I therefore limited the test to my favorite samples (i.e. easier to ABX), and maintained two encoders only:
- GT3b2 at -q 6.00
- aoTuV MEGAMIX at -q 6.00
RESULTS
(http://www.foobar2000.net/divers/tests/2004.06/results_megamix_q6.png)
CONCLUSIONS:
• GT3b2 is still superior to aoTuV MEGAMIX on micro-attacks samples.
• on very sharp attacks, the MEGAMIX addition is perfectly implemented: quality is at least eaqual to GT3b2,
and sometimes better. aoTuV tunings and GT3b2 tunings are apparently cumulative for this kind of sample.
• there's apparently more noise problem with GT3b2 than aoTuV MEGAMIX: audible with
Sarabande (harpsichord),
Danse Hongroise (piano),
Atem-Lied (ringing).
• I must add that the "hollow" / "distorted" sound of aoTuV (MEGAMIX or not), audible at -q 5,00 with
Shostakovich_sym14, is also audible at -q 6,00 with the same encoder. GT3b2 seems to be free of this problem, and sound also better than the reference! (I explained the reasons on the log file).
LOG files are available here:
http://www.foobar2000.net/divers/tests/200...EGAMIX%20-Q5.7z (http://www.foobar2000.net/divers/tests/2004.06/MEGAMIX%20-Q5.7z)
http://www.foobar2000.net/divers/tests/200...EGAMIX%20-Q6.7z (http://www.foobar2000.net/divers/tests/2004.06/MEGAMIX%20-Q6.7z)
Thank you for the tests, Tang. They are very detailed and informative.
Thank you for the tests, Tang. They are very detailed and informative.
Joke?
Thank you for the tests, Tang. They are very detailed and informative.
Joke?
Not at all. I quite like how you put a description for each sample too, whether it is a microattack or sharp-attack. These are the things I often like to know when comparing the numbers.
Thank you for the tests, Tang. They are very detailed and informative.
Joke?
Not at all. I quite like how you put a description for each sample too, whether it is a microattack or sharp-attack. These are the things I often like to know when comparing the numbers.
Seems to be some misunderstanding...
I AM NOT THE AUTHOR OF THESES TESTS Quantum-Knot!!!
It's Guruboolez's works... i've just pasted his "formated text" cause he had some connection problem with HA...
I've edited the previous quote to make the "introducting notice" more evident...
Oups, I'm connected again....
I'd like to thanks Tang for posting my tests. I spent a lot of time yesterday, and I was very angry to see that HA wasn't accessible on my computer.
APPENDIX (2)
I have also noticed two times the same "hollow sound" artifact, audible with aoTuV based encoders, but not with GT3b2: with clapping and with shostakovich_sym14.
I think I know exactly what you mean. I noticed this artifact with aoTuV on "The Frog Princess" by The Divine Comedy
Casanova...but on low quality levels (< -q 2). I didn't notice it at -q 6. The artifact sounds like a band of frequencies is missing, the middle and lower treble in particular. It's like someone took a vacuum and sucked out some of the music!
I've updated my chart above to include two new songs. A few people had mentioned that my average bitrates was lower than what they were getting on average, as was the bitrate differential between gt3b2 and non-preecho tunings (xiph.org and aotuv). I went back to look at the bitrate distribution on my whole collection of encoded files (about 200 albums with gt3b2 at -q 6), and found that while most fell in a bell curve, as expected, around 190kbps, there were a fair number of outliers more than 70kbps over the average. On the other hand, there were no outliers below 60kbps below the average. None of these upper-outliers were represented in my test group, so I've added a couple: "Nannou" off of Aphex Twin - "Windowlicker" and "(-) Ions" off of "Tool - AEnema".
True to my hypothesis, for these "bitrate monsters", preecho tuned encoders bloat proportionally more than the average song. For example:
aotuv aotuv
-q 6.0 xiph.org aotuv qk32 gt3b2 +qk32 +gt3b2 megamix
Ions 271.0 331.7 270.1 379.6 331.7 405.7 405.7
Nannou 228.8 252.4 227.7 277.8 252.4 286.4 286.4
Average 183.5 187.5 182.9 195.0 187.5 195.9 195.9
For the average song, gt3b2 averages about 6.2% higher than xiph.org and 4% higher than aotuv. For something pathological like Ions, it's 40% and 14% vs. xiph.org an aotuv respectively. Even for Nannou (which is a lot less weird than Ions, and a beautiful little song), it's a 21% or 10% increase.
In the end, it doesn't change my numbers all that much. For example, the -q 6 equivalent for aotuv goes from 6.23 to 6.29. It's pretty vain of me to suggest I have 3 significant figures here anyway. It did make aotuv+qk32 (and megamix) an even bigger outlier in the 2 <= -q < 4 range, but it was already too high in that range to get a good comparison anyway.
If someone is looking to do a listening test, I suggest they use the new numbers (the lower-right table), but the difference between the new and old numbers is not big enough to be signfiicant.
I think I know exactly what you mean. I noticed this artifact with aoTuV on "The Frog Princess" by The Divine Comedy Casanova...but on low quality levels (< -q 2). I didn't notice it at -q 6. The artifact sounds like a band of frequencies is missing, the middle and lower treble in particular. It's like someone took a vacuum and sucked out some of the music!
At those -q levels the "hollow sound" artifact is a pretty common one with xiph.org vorbis too. It was discussed some in the aftermath of the 64k test. It's a bit unsettling that Guruboolez heard it at high bitrates too. I wonder what it is in gt3b2's preecho tunings that fixes it...
http://www.hydrogenaudio.org/forums/index....ndpost&p=136491 (http://www.hydrogenaudio.org/forums/index.php?showtopic=13464&view=findpost&p=136491)
http://www.hydrogenaudio.org/forums/index....ndpost&p=136490 (http://www.hydrogenaudio.org/forums/index.php?showtopic=7197&view=findpost&p=136490)
Thank you for the tests, Tang. They are very detailed and informative.
Joke?
Not at all. I quite like how you put a description for each sample too, whether it is a microattack or sharp-attack. These are the things I often like to know when comparing the numbers.
Seems to be some misunderstanding...
I AM NOT THE AUTHOR OF THESES TESTS Quantum-Knot!!!
It's Guruboolez's works... i've just pasted his "formated text" cause he had some connection problem with HA...
I've edited the previous quote to make the "introducting notice" more evident...
whoops. I didn't pay attention to that. The numbers distracted my eyes. Well, thanks for being the messenger and thanks to guruboolez for his meticulous test.
question about vorbis megamix
between q 1 to q 2 and between q 4 to q 5 (e.g. q 1.50 and q 4.75) , what tuning encoder is chosen?
q 1 to q 2 -> aoTuV beta 2? QuantumKnot tune beta 3.2 + aoTuV beta 2? or both marge?
q 4 to q 5 -> QuantumKnot tune beta 3.2 + aoTuV beta 2? Galf Tuned 3 beta 2 + aoTuV beta 2? or both marge?
question about vorbis megamix
between q 1 to q 2 and between q 4 to q 5 (e.g. q 1.50 and q 4.75) , what tuning encoder is chosen?
q 1 to q 2 -> aoTuV beta 2? QuantumKnot tune beta 3.2 + aoTuV beta 2? or both marge?
q 4 to q 5 -> QuantumKnot tune beta 3.2 + aoTuV beta 2? Galf Tuned 3 beta 2 + aoTuV beta 2? or both marge?
between q 1 and 2, I expect it to be partial QKTune with the full tunings coming into effect at 2. Same for q 4 to 5 as well (partial QKTune, partial GT3b2).
So, is it safe to assume that the new Vorbis Megamix can be used outside of testing?
question about vorbis megamix
between q 1 to q 2 and between q 4 to q 5 (e.g. q 1.50 and q 4.75) , what tuning encoder is chosen?
q 1 to q 2 -> aoTuV beta 2? QuantumKnot tune beta 3.2 + aoTuV beta 2? or both marge?
q 4 to q 5 -> QuantumKnot tune beta 3.2 + aoTuV beta 2? Galf Tuned 3 beta 2 + aoTuV beta 2? or both marge?
between q 1 and 2, I expect it to be partial QKTune with the full tunings coming into effect at 2. Same for q 4 to 5 as well (partial QKTune, partial GT3b2).
i understood.
thank you, QuantumKnot. you are great!!!!
So, is it safe to assume that the new Vorbis Megamix can be used outside of testing?
I think more listening tests on a range of q values would be needed as Vorbis Megamix may still regress in some ranges for some samples. But if you feel adventurous, you can always try it (on stuff that you can re-encode later if needed)
I've finished up some tests feature aoTuVb2 -q 4.47 and Megamix -q 4.00. I cut other encoders down to spare time.
(http://cyberquebec.ca/harashin/pre-echo_test_q4.png)
Conclusion: I haven't been able to find any regression(i.e. additional hiss) with Megamix at least for these samples at -q4.
I'm impressed with Megamix especially for harpsichord sample(Sarabande, not perfect though).
Anyhow, since I still have trouble with HA's uploader, I uploaded Ravel_Alborada.flac (http://cyberquebec.ca/harashin/Ravel_Alborada.flac) at my hosting space as well. This is a kind of "sharp attack" sample.
pre-echo_q4_results.zip (http://cyberquebec.ca/harashin/pre-echo_q4_results.zip)
I've compiled a linux static megamix binary if anybody wants to try this on linux:
http://www.caddr.com/oggenc (http://www.caddr.com/oggenc)
Many thanks to all of you for your work on vorbis!
Considering this tests and my personal ones, it appears that the megamix is indeed not a regression comparate to oaTuVb2 :-)
It also seems that oaTuVb2 + QK is better than oaTuVb2 + GT3b2 (and with less bitrate) ... I've donne several tests at -q5 and oaTuVb2 + QK really works great. I can't ear any hiss problem with it :-) So I really would like to test oaTuVb2 + QK tuned for all quality levels...
oaTuVb2 + QK at -q7 may be transparent for everybody and for everything and with a little bit more tuning on microattacks, ogg/vorbis will be able to replace MPC ;-)
QuantumKnot, what do you think about that ?
I've compiled a linux static megamix binary if anybody wants to try this on linux:
http://www.caddr.com/oggenc (http://www.caddr.com/oggenc)
Thanx kuniklo! Goo Goo Dolls' Gutterflower sounds great at q7.
Do you think we can get an oggdropXPd build using the megamix code?
Considering this tests and my personal ones, it appears that the megamix is indeed not a regression comparate to oaTuVb2 :-)
It also seems that oaTuVb2 + QK is better than oaTuVb2 + GT3b2 (and with less bitrate) ... I've donne several tests at -q5 and oaTuVb2 + QK really works great. I can't ear any hiss problem with it :-) So I really would like to test oaTuVb2 + QK tuned for all quality levels...
oaTuVb2 + QK at -q7 may be transparent for everybody and for everything and with a little bit more tuning on microattacks, ogg/vorbis will be able to replace MPC ;-)
QuantumKnot, what do you think about that ?
Given some time, I can probably tune all the way up to q 10 but my ears aren't that sensitive to pre-echo at these high bitrates. It happens that q 2, 3, 4, 5 had serious issues with pre-echo, to the point where I was able to discern quite easily. Any type of tuning I do at q > 6 would be similar to GT3b2, but I'll be aiming for less bits. We'll see whether I have some spare time. By the time I do, aoyumi or Monty will probably have a better version out. hehe
As for Vorbis replacing MPC on things like microattacks, I doubt we'll see that in the foreseeable future.
Do you think we can get an oggdropXPd build using the megamix code?
Sure can. Sorry, I've been a bit distracted of late with system rebuild and heavy video encoding. Give me a few hours and I'll see what I can do.
Do you think we can get an oggdropXPd build using the megamix code?
At Rarewares now in P3 and P4 versions.
Lev loves John33
Thanks for the build. It works fine.
I want to do the download.
But, the connection is failed at a http://www.rarewares.org/ (http://www.rarewares.org/) site.
I am expecting the Oggenc2.3 P4.
Because the oggenc-megamix.exe is too slow on my computer
I am expecting the Oggenc2.3 P4.
Because the oggenc-megamix.exe is too slow on my computer
OK, I hear you!! I'll do it soon.
Thank you!!!!
I am expecting the Oggenc2.3 P4.
Because the oggenc-megamix.exe is too slow on my computer
Done. ICL 7.1 compiles now at Rarewares.
I get it. it works fine.
Thank you!!
I am expecting the Oggenc2.3 P4.
Because the oggenc-megamix.exe is too slow on my computer
Done. ICL 7.1 compiles now at Rarewares.
The "about" box reports vorbis Post Release 1.0.1 CVS with OggDrop Megamix. Didn't know if that's OK.
If you ever get bored john33 you can update the quality management mode slider control to go down to -2 since the current batch of encoders recognize this quality level.
I am expecting the Oggenc2.3 P4.
Because the oggenc-megamix.exe is too slow on my computer
Done. ICL 7.1 compiles now at Rarewares.
The "about" box reports vorbis Post Release 1.0.1 CVS with OggDrop Megamix. Didn't know if that's OK.
Yeah, that's OK. I was in a hurry and forgot to update the box. I assume the vendor string says the right thing?
I'll update the box and reload in the next couple of hours. Thanks for letting me know.
Why the song using oggdropXPd is not the same size as the one using Oggenc2.3 P4 or oggenc-megamix.exe.
I see the song's size with Foobar.
The "about" box reports vorbis Post Release 1.0.1 CVS with OggDrop Megamix. Didn't know if that's OK.
I've uploaded new compiles. No code changes, just the "About" box title corrected.
Why the song using oggdropXPd is not the same size as the one using Oggenc2.3 P4 or oggenc-megamix.exe.
I see the song's size with Foobar.
I believe 'oggenc-megamix' is a MSVC compile, so I would expect a difference with a different math lib. However, I can't quite see, ATM, why there is a difference with the other two. On a 5.5MB encode at '-q 5', there is a difference of 93 bytes. Entirely inconsequential, but I'd like to establish why!!
That MegaMix is (going to be) a wonderful thing guys.
BTW I want to remember you that aoTuV has a -q-2 setting, so please include it in further comparisons and stuff...
Why the song using oggdropXPd is not the same size as the one using Oggenc2.3 P4 or oggenc-megamix.exe.
I see the song's size with Foobar.
I believe 'oggenc-megamix' is a MSVC compile, so I would expect a difference with a different math lib. However, I can't quite see, ATM, why there is a difference with the other two. On a 5.5MB encode at '-q 5', there is a difference of 93 bytes. Entirely inconsequential, but I'd like to establish why!!
Yeah, it's an MSVC compile (with default settings).
One thing to note is that, since the codebase is essentially aoTuV beta 2, then using bitrate management is not advisable.
Is MegaMix better than aoTuVb2???
I mean:Is MegaMix better than aoTuVb2 at q4?
I mean:Is MegaMix better than aoTuVb2 at q4?
Megamix will tend to have less pre-echo than aoTuV beta 2. Try it on the castanets sample
Whether it is better than aoTuV beta 2 at q 4, check out the listening tests above. Or you can try listening for yourself.
oggenc2.exe megamix crashes everytime when a filename that does not exist is given instead of displaying an "error:file does not exist" or whatever. No big deal though, I just thought you'd know.
oggenc2.exe megamix crashes everytime when a filename that does not exist is given instead of displaying an "error:file does not exist" or whatever. No big deal though, I just thought you'd know.
Both the P3 and P4 compiles behave correctly here.
Both the P3 and P4 compiles behave correctly here.
I'm using the P3 compile since I'm on a Celeron 900 here. It encodes just fine though. Doesn't anybody else have this kind of "problem"?
Why the song using oggdropXPd is not the same size as the one using Oggenc2.3 P4 or oggenc-megamix.exe.
I see the song's size with Foobar.
I believe 'oggenc-megamix' is a MSVC compile, so I would expect a difference with a different math lib. However, I can't quite see, ATM, why there is a difference with the other two. On a 5.5MB encode at '-q 5', there is a difference of 93 bytes. Entirely inconsequential, but I'd like to establish why!!
Yeah, it's an MSVC compile (with default settings).
One thing to note is that, since the codebase is essentially aoTuV beta 2, then using bitrate management is not advisable.
I'm sorry if this is off-topic, but neither your MSVC compile nor John's ICL compile accepts wild card (like *.wav). They spit out an error message 'ERROR: Cannot open input file "*.wav": No such file or directory
'.
Aoyumi's compile of his AoTuV b2 on the other hand accepts wild card with no problem, and I love it Is that a matter of compile options or the compiler difference?
I'm having trouble compiling megamix (http://www.rarewares.org/quantumknot/vorbis-megamix-src.zip) on Linux.
The code uses some WIN32ism's, this patch is needed to solve this:
--- vorbis-megamix/examples/encoder_example.c.orig 2004-04-21 22:59:10.000000000 +0200
+++ vorbis-megamix/examples/encoder_example.c 2004-07-04 17:55:18.105580606 +0200
@@ -75,7 +75,7 @@ signed char readbuffer[READ*4+44]; /* ou
puts("usage: encoder_example -q4 filename.wav");
exit(1);
}
- if( !strnicmp(argv[1], "-q", 2) ){
+ if( !strncasecmp(argv[1], "-q", 2) ){
if( strlen(argv[1]) > 2 ){
qnum = atof(argv[1]+2);
}
@@ -114,9 +114,9 @@ signed char readbuffer[READ*4+44]; /* ou
verify that it matches 16bit/stereo/44.1kHz. This is just an
example, after all. */
- DWORD footer = 0; //+
- DWORD sa_rate = 44100; //+
- WORD channel; //+
+ unsigned long footer = 0; //+
+ unsigned long sa_rate = 44100; //+
+ unsigned short channel; //+
struct stat sbuf; //+
stat(argv[qx], &sbuf); //+
Furthermore I'm having trouble running the configure script, it just spits out:
: bad interpreter: No such file or directory
I can solve this problem by running autoconf, but then the configure script errors with
configure: error: cannot run /bin/sh ./config.sub
I'm having trouble compiling megamix (http://www.rarewares.org/quantumknot/vorbis-megamix-src.zip) on Linux.
The code uses some WIN32ism's, this patch is needed to solve this:
--- vorbis-megamix/examples/encoder_example.c.orig 2004-04-21 22:59:10.000000000 +0200
+++ vorbis-megamix/examples/encoder_example.c 2004-07-04 17:55:18.105580606 +0200
@@ -75,7 +75,7 @@ signed char readbuffer[READ*4+44]; /* ou
puts("usage: encoder_example -q4 filename.wav");
exit(1);
}
- if( !strnicmp(argv[1], "-q", 2) ){
+ if( !strncasecmp(argv[1], "-q", 2) ){
if( strlen(argv[1]) > 2 ){
qnum = atof(argv[1]+2);
}
@@ -114,9 +114,9 @@ signed char readbuffer[READ*4+44]; /* ou
verify that it matches 16bit/stereo/44.1kHz. This is just an
example, after all. */
- DWORD footer = 0; //+
- DWORD sa_rate = 44100; //+
- WORD channel; //+
+ unsigned long footer = 0; //+
+ unsigned long sa_rate = 44100; //+
+ unsigned short channel; //+
struct stat sbuf; //+
stat(argv[qx], &sbuf); //+
Furthermore I'm having trouble running the configure script, it just spits out:
: bad interpreter: No such file or directory
I can solve this problem by running autoconf, but then the configure script errors with
configure: error: cannot run /bin/sh ./config.sub
The problem is that there are windows-style linefeeds in several of the config files/scripts.
I finally got this all working. You can download a linux ogg megamix from here:
http://www.caddr.com/oggenc (http://www.caddr.com/oggenc)
Oh, of course, I had not thought about this. It's working now after a bit of dos2unix :-) Thank you. It would be a good idea that the authors would provide a working tarball for Linux together with the windows binaries.
The "about" box reports vorbis Post Release 1.0.1 CVS with OggDrop Megamix. Didn't know if that's OK.
I've uploaded new compiles. No code changes, just the "About" box title corrected.
Starting "mass-encode" for my iRiver
I'm sorry if this is off-topic, but neither your MSVC compile nor John's ICL compile accepts wild card (like *.wav). They spit out an error message 'ERROR: Cannot open input file "*.wav": No such file or directory
'.
Aoyumi's compile of his AoTuV b2 on the other hand accepts wild card with no problem, and I love it Is that a matter of compile options or the compiler difference?
Yes, MSVC compiles don't support wildcards. The reason aoyumi's compile works is because he used mingw32, where wildcards are known to work with the oggenc's. I can do a mingw32 compile but John33 can probably doing it quicker than me.
Oh, of course, I had not thought about this. It's working now after a bit of dos2unix :-) Thank you. It would be a good idea that the authors would provide a working tarball for Linux together with the windows binaries.
I usually do development in Linux but due to lack of time (plus the larger userbase of Windows users), I just did the merging in MSVC and make the win32 binaries. I forgot about this problem. My apologies
I'm sorry if this is off-topic, but neither your MSVC compile nor John's ICL compile accepts wild card (like *.wav). They spit out an error message 'ERROR: Cannot open input file "*.wav": No such file or directory
'.
Aoyumi's compile of his AoTuV b2 on the other hand accepts wild card with no problem, and I love it Is that a matter of compile options or the compiler difference?
Yes, MSVC compiles don't support wildcards. The reason aoyumi's compile works is because he used mingw32, where wildcards are known to work with the oggenc's. I can do a mingw32 compile but John33 can probably doing it quicker than me.
QK, thanks for your response. I was just wondering what causes the difference. Now I know that it is the compiler you choose to use that allows or disallows wild card use.
By the way, there is no need to re-compile it with MINGW32 unless other people want it. I can always use FOR and DO if I need to (for %a in (*.wav) do oggenc-megamix -q 6 "%a"). I would much rather that you spend your spare time working on quality improvement than making another compile that people may or may use Thanks again.
Yes, MSVC compiles don't support wildcards.
Unless you link against "setargv.obj". It's not terribly well documented, but it is in there (somewhere). Hope this helps.
Yes, MSVC compiles don't support wildcards. The reason aoyumi's compile works is because he used mingw32, where wildcards are known to work with the oggenc's. I can do a mingw32 compile but John33 can probably doing it quicker than me.
For anyone who is interested, MinGW32 (GCC 3.3.3) compiles of oggenc2.3 MegaMix are now available at Rarewares.
I appreciate what you've done.
I thank about updating at a 'http://homepage.ntlworld.com/jfe1205/' homepage.
Just to want to let everyone know that the GT3b2 tunings in Vorbis Megamix are not complete. Many thanks to Aoyumi for finding this. I think I will incorporate this into Megamix 2 which will be a merge with 1.1RC1. The value under question affects block switching only.
Might this additional code improve performances with micro-attacks samples? What do you think? Difference between megamix and GT3b2 are surprising.
For exemple: vorbis is adding something comparable to a "noise curtain" on the creaking.wav sample, and the length of this curtain is much shorter with GT3b2 than with aoTuV/megamix:
http://www.foobar2000.net/divers/tests/200...gamix_aoTuV.gif Wrong link: see two post below.
Might this additional code improve performances with micro-attacks samples? What do you think? Difference between megamix and GT3b2 are surprising.
For exemple: vorbis is adding something comparable to a "noise curtain" on the creaking.wav sample, and the length of this curtain is much shorter with GT3b2 than with aoTuV/megamix:
http://www.foobar2000.net/divers/tests/200...gamix_aoTuV.gif (http://www.foobar2000.net/divers/tests/2004.06/diff_megamix_aoTuV.gif)
[a href="index.php?act=findpost&pid=225718"][{POST_SNAPBACK}][/a]
Yes, it's mostly likely due to this. I apologise for overlooking this when I did the merge. I'll try to be a bit more careful this time.
[span style='font-size:14pt;line-height:100%']Wrong link.[/span]
Correct links are:
http://www.foobar2000.net/divers/tests/200...iff_ref_GT3.gif (http://www.foobar2000.net/divers/tests/2004.06/diff_ref_GT3.gif)
http://www.foobar2000.net/divers/tests/200...ref_megamix.gif (http://www.foobar2000.net/divers/tests/2004.06/diff_ref_megamix.gif)
First one correspond to a GT3 / original comparison
Second one to megamix / original comparison