Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Vorbis 1.1 vs Vorbis aoTuVb3 (Read 12726 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Vorbis 1.1 vs Vorbis aoTuVb3

I have done a few tests between Vorbis 1.1 and Vorbis aoTuVb3 at around 64kbps, to possibly help with selection between the two in the upcoming multiformat 64kbps test.

First test: kraftwerk

Code: [Select]
ABC/HR Version 1.0, 6 May 2004
Testname: kraftwerk

1R = kraftwerk-Vorbis1.1_Q0.wav
2L = kraftwerk-VorbisaoTuVb3_Q0.wav

---------------------------------------
General Comments:

---------------------------------------
1R File: kraftwerk-Vorbis1.1_Q0.wav
1R Rating: 3.0
1R Comment: 
---------------------------------------
2L File: kraftwerk-VorbisaoTuVb3_Q0.wav
2L Rating: 2.9
2L Comment: 
---------------------------------------
ABX Results:
Original vs kraftwerk-Vorbis1.1_Q0.wav
    10 out of 10, pval < 0.001
Original vs kraftwerk-VorbisaoTuVb3_Q0.wav
    10 out of 10, pval < 0.001
kraftwerk-Vorbis1.1_Q0.wav vs kraftwerk-VorbisaoTuVb3_Q0.wav
    22 out of 30, pval = 0.008


Here both encoded samples are very easy to ABX.  It is very difficult to ABX between the two encoded samples, but aoTuVb3 seemed to add a bit of extra noise (very very small amount) a few seconds into the sample.

Second test: SinceAlways

Code: [Select]
ABC/HR Version 1.0, 6 May 2004
Testname: SinceAlways

1L = SinceAlways-VorbisaoTuVb3_Q0.wav
2L = SinceAlways-Vorbis1.1_Q0.wav

---------------------------------------
General Comments:

---------------------------------------
1L File: SinceAlways-VorbisaoTuVb3_Q0.wav
1L Rating: 3.0
1L Comment: HF Boost, "warped" sound on hi-hats
---------------------------------------
2L File: SinceAlways-Vorbis1.1_Q0.wav
2L Rating: 4.0
2L Comment: HF Boost and distortion on crash cymbal.
---------------------------------------
ABX Results:
Original vs SinceAlways-VorbisaoTuVb3_Q0.wav
    10 out of 10, pval < 0.001
Original vs SinceAlways-Vorbis1.1_Q0.wav
    13 out of 14, pval < 0.001
SinceAlways-VorbisaoTuVb3_Q0.wav vs SinceAlways-Vorbis1.1_Q0.wav
    18 out of 20, pval < 0.001


A bit more interesting here.  aoTuVb3 seems to be easier to ABX vs the original.  Easier to ABX between the two encoded samples as they have artifacts in difference places.  1.1 has larger HF boost on the crash cymbals, but aoTuVb3 has strange "warped" sound on some of the hi-hats.

Overall, Vorbis 1.1 seems to be winning.

More tests to come...
 

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #1 –
Good to read that. I thought Aotuv was only really tuned for higher bitrates? That seems to confirm it, but I am still a little surprised.

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #2 –
Aoyumi worked on -q0...-q-1 and created -q-2 preset. AoTuV can't be tuned for high bitrate only

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #3 –
Quote
Aoyumi worked on -q0...-q-1 and created -q-2 preset. AoTuV can't be tuned for high bitrate only
[a href="index.php?act=findpost&pid=285102"][{POST_SNAPBACK}][/a]


Hmm yeah good point. Merci vilmal min Hèrr, or however they say it in Elsässich.

The reason I was asking was that certain codecs can be tuned for lower bitrates: HE-AAC, mp3PRO, or the early days of Vorbis... or does HE-AAC actually perform better at high bitrates too ( < 192 Kbps average)?

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #4 –
SBR is only good for bitrates up to 96 kbps. LC AAC wins it at higher bitrates.

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #5 –
Quote
or does HE-AAC actually perform better at high bitrates too ( < 192 Kbps average)?
[a href="index.php?act=findpost&pid=285129"][{POST_SNAPBACK}][/a]

No (not with current encoders).



Latexxx was faster.

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #6 –
I am glad that you started this thread Phreakazoid and hope that other tests will follow soon, so that the best Vorbis encoder will be used for the test.

While testing should focus on 1.1 and AoTuV3, you might want to give Archer a try, but only if you have time.

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #7 –
I've done another test, this time with Archer RC4 in the mix.  One of the listed features of the aoTuVb3 release was improved pre-echo handling, so I've used the castanets sample as a test of this behaviour.

Results:

Code: [Select]
ABC/HR Version 1.0, 6 May 2004
Testname: castanets

1R = castanets-VorbisArcherRC4_Q0.wav
2R = castanets-Vorbis1.1_Q0.wav
3L = castanets-VorbisaoTuVb3_Q0.wav

---------------------------------------
General Comments:

---------------------------------------
1R File: castanets-VorbisArcherRC4_Q0.wav
1R Rating: 1.5
1R Comment: Smearing, noisy, distortion.
---------------------------------------
2R File: castanets-Vorbis1.1_Q0.wav
2R Rating: 1.0
2R Comment: Smearing, noisy, distortion.
---------------------------------------
3L File: castanets-VorbisaoTuVb3_Q0.wav
3L Rating: 1.5
3L Comment: Smearing, noisy, distortion.
---------------------------------------
ABX Results:
Original vs castanets-VorbisArcherRC4_Q0.wav
    10 out of 10, pval < 0.001
Original vs castanets-Vorbis1.1_Q0.wav
    10 out of 10, pval < 0.001
Original vs castanets-VorbisaoTuVb3_Q0.wav
    10 out of 10, pval < 0.001
castanets-VorbisArcherRC4_Q0.wav vs castanets-Vorbis1.1_Q0.wav
    26 out of 33, pval < 0.001
castanets-VorbisArcherRC4_Q0.wav vs castanets-VorbisaoTuVb3_Q0.wav
    4 out of 10, pval = 0.828
castanets-Vorbis1.1_Q0.wav vs castanets-VorbisaoTuVb3_Q0.wav
    18 out of 21, pval < 0.001


All of the encoded samples sounded pretty terrible (as was expected), but it seems that aoTuVb3 and Archer RC4 both improved a little over 1.1.  In the Vorbis 1.1 sample, there was a bit of extra noise on the castanet rattle, compared to aoTuVb3 and Archer RC4.

For this test... equal winners are aoTuVb3 and Archer RC4.

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #10 –
I released the newest version of aoTuV pre-beta4 [20050403]. Unless I or other someone find a big problem, there will not be formal beta4 and a big difference. 

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #12 –
Would it be possible to get the source in order to compile it on linux? I guess it's not a problem as the source of the experimental version is provided on the site.

Thanks!

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #13 –
Quote
Would it be possible to get the source in order to compile it on linux? I guess it's not a problem as the source of the experimental version is provided on the site.

Thanks!
[a href="index.php?act=findpost&pid=288568"][{POST_SNAPBACK}][/a]


The reason I do not upload a source code is that it is not adjusting more than q1 with having scribbled the comment.

About the experiment version, the version to which I open the source code is performing minimum tests and minimum checks. 

please wait.

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #14 –
Again, I would like to ask everyone who is interested to conduce more Vorbis listening tests. The 64 kbps multi-format test will most probably start late April or early May. Tiger is supposed to come out on April 29th and with it the Apple HE-AAC encoder.

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #16 –
I uploaded oggenc(win32) of pre-beta4 now.
It includes the new tuning by all the bit rates. 

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #17 –
Anything about the date of the final release of Aotuv Beta 4, Aoyumi?

When is going to be a "beta-stable"?

And just for curiosity, it have really big important tunings? like the tunings from 1.0.1 to the beta2... for being the next vorbis release (1.2.0?). 

And thanks for you great work, I really (and many others) appreciate it. 
JorSol
aoTuVb5 -q4

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #18 –
Quote
Anything about the date of the final release of Aotuv Beta 4, Aoyumi?
When is going to be a "beta-stable"?

The release time of beta4 is dependent on the degree of advance of adjustment and a test.
I want to release by the beginning in May.

Quote
And just for curiosity, it have really big important tunings? like the tunings from 1.0.1 to the beta2... for being the next vorbis release (1.2.0?). 

Yes, it is a change important for me as a user at least. New code and tunings are included in beta 3/4. Although beta3 solved some problems, there was a case where the problem of channel coupling was highlighted as a result. And beta4 was made in order to solve the problem of beta3.
For example, beta3 makes noise control more dynamic, in order to reduce pre-echo. It affected control of channel coupling, therefore the problem was produced.  By correction of this channel coupling processing, those problems are solved in general.

About an official vorbis release, nothing can be said from me. For the moment, I am not involving.

[EDIT] TYPO

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #19 –
Aoyumi:
As far as I know, Monty loves your job, he seems to rely on you to keep the house during the time he will code for vorbis II nowadays ... last time I saw him he was already planning to test the beta3 new low bitrates mode ... so there is no doubt he will swallow beta4 with 1.1 post svn if ABXing test are good ... specially if there is a 64kbps listening test & aob4 wins ... not only Monty, but other Xiph dev seems to push Monty to swallow aotuv quick ... anyway we'll remind him of you if he ever forgets

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #20 –
Quote
specially if there is a 64kbps listening test & aob4 wins
[a href="index.php?act=findpost&pid=290993"][{POST_SNAPBACK}][/a]


Who says that AoTuV4 will be tested? I am still waiting for some ABX tests to decide which Vorbis encoder to be featured.

So far, I have three test results: AoTuV won once, Xiph twice.

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #21 –
Sebastian Mares:
yes indeed  ... I was just planning the best that can happen ... no one says aob4 will win the overall test too, if selected

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #22 –
I'm currently testing xiph 1.1.0 vs archer RC4 vs aoTuv PB4 to help decide which one should be used in the upcoming 64kbps tests.
I just wanted to post a preliminary report after 18 of 37 samples. puh this takes time
I used foobar to extract lossless sample clips to wav, and decode all vorbis clips back to wav. "foobar2000 v0.8.3 unicode version Compiled on: Jun 28 2004"
Gear: Sennheiser HD600 driven by asus P4P800 mainboard onboard audio (analog devices soundmax)
 
So far, I've only found 2 samples where I could abx the difference between vorbis tunings/versions.

First one is "Autechre - Gantz Graf (02)"
http://moozooh.nm.ru/samples_for_ha/ae_gg02.ape

ABX Results:
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\2.wav.wav
    8 out of 8, pval = 0.004
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\2.wav
    8 out of 8, pval = 0.004
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\2.wav
    8 out of 8, pval = 0.004
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\2.wav.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\2.wav
    8 out of 8, pval = 0.004
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\2.wav.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\2.wav
    8 out of 8, pval = 0.004

easy to abx. transient smear, stereo colapse. xiph/archer got alot of "treble boost" effect. aotuv pb4 much closer to the original.  xiph and archer sounds identical.


The other one is "Deep_Purple___When_A_Blind_Man_Cries"
http://www.hydrogenaudio.org/forums/index....pe=post&id=1428

ABX Results:
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\10.wav.wav
    16 out of 16, pval < 0.001
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\10.wav
    17 out of 17, pval < 0.001
Original vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\10.wav
    16 out of 16, pval < 0.001
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\10.wav.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\10.wav
    23 out of 32, pval = 0.010
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\aoTuV_pre_b4_20050412\10.wav.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\10.wav
    25 out of 32, pval = 0.001
C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\oggdropXPdV1.8.3-generic\10.wav vs C:\_hydrogen_audio\ogg_test__xiph1.1.0_vs_aotuvPB4_vs_archerRC4\OggEnc_SSE_20050320ArcherRC4\10.wav
    12 out of 12, pval < 0.001

only slight difference between xiph and archer, but I prefer xiph. tiny bit less flange effect. 12 out of 12 might seem it was easy, but it was very hard to abx.
It could be that the flanging was not in sync, and made it possible for me to tell them apart. overall sound was very close.
aotuvPB4 worst of them. Much easier to abx between aotuv and xiph/archer than between xiph and archer.
Got some spells of listening fatigue, that's why they go up to 32tests.

I would say the win by aoTuv PB4 on the autechre transient test was bigger than the win xiph 1.1.0 got on the deep purple track.

It's obvious to me that my ears are more sensitive to some effects than others. I hope more people can do tests, and atleast do tests on these two tracks to verify my findings.
I'll post again when I've completed all the 37 sample clips

Vorbis 1.1 vs Vorbis aoTuVb3

Reply #24 –
I did some comparison between 1.1 and aoTuV pb4, which was very hard for me. I wanted to use samples employed by the former 64Kbps listening test, but I could not find the conspicuous difference between 1.1 and aoTuV pb4. So I gave up using these samples and switched to my daily listening samples. Then I found the following samples from several albums that have differences among the two encoders.

"Take Me Out", "Franz Ferdinand" by Franz Ferdinand [sample]
Code: [Select]
ABC/HR Version 1.0, 6 May 2004
Testname: Take Me Out

1R = C:\test\TakeMeOut-1.1.wav
2R = C:\test\TakeMeOut-pb4.wav

---------------------------------------
General Comments:
Sample 1 has slightly louder open hi-hat shots.
---------------------------------------
1R File: C:\test\TakeMeOut-1.1.wav
1R Rating: 3.0
1R Comment: 
---------------------------------------
2R File: C:\test\TakeMeOut-pb4.wav
2R Rating: 4.0
2R Comment: 
---------------------------------------
ABX Results:
C:\test\TakeMeOut-1.1.wav vs C:\test\TakeMeOut-pb4.wav
    13 out of 18, pval = 0.048


"Mint", "Even So" by Bonnie Pink [sample]
Code: [Select]
ABC/HR Version 1.0, 6 May 2004
Testname: Mint

1L = C:\test\Mint-1.1.wav
2R = C:\test\Mint-pb4.wav

---------------------------------------
General Comments:
Sample 2 is better than sample 1 for the crash cymbal.
---------------------------------------
1L File: C:\test\Mint-1.1.wav
1L Rating: 2.0
1L Comment: 
---------------------------------------
2R File: C:\test\Mint-pb4.wav
2R Rating: 1.5
2R Comment: 
---------------------------------------
ABX Results:
C:\test\Mint-1.1.wav vs C:\test\Mint-pb4.wav

    9 out of 11, pval = 0.033


"Genjitsu ni oite", "Kyoiku" by Tokyo Jihen [sample]
Code: [Select]
ABC/HR Version 1.0, 6 May 2004
Testname: Genjitsu ni oite

1R = C:\test\genjitsu-ni-oite-pb4.wav
2R = C:\test\genjitsu-ni-oite-1.1.wav

---------------------------------------
General Comments:
Sample 2 has muddy and unstable piano tone. Sample 1 is much better than sample 2.
---------------------------------------
1R File: C:\test\genjitsu-ni-oite-pb4.wav
1R Rating: 3.5
1R Comment: 
---------------------------------------
2R File: C:\test\genjitsu-ni-oite-1.1.wav
2R Rating: 2.0
2R Comment: 
---------------------------------------
ABX Results:
C:\test\genjitsu-ni-oite-pb4.wav vs C:\test\genjitsu-ni-oite-1.1.wav
    5 out of 5, pval = 0.031


"Fuyu no sanatorium", "Hajime Chitose" by Hajime Chitose [sample]
Code: [Select]
ABC/HR Version 1.0, 6 May 2004
Testname: Fuyu no sanatorium

1L = C:\test\fuyu-no-sanatorium-1.1.wav
2L = C:\test\fuyu-no-sanatorium-pb4.wav

---------------------------------------
General Comments:
Sample 2 is better than Sample 1 for the guitar arpeggio.
---------------------------------------
1L File: C:\test\fuyu-no-sanatorium-1.1.wav
1L Rating: 2.0
1L Comment: 
---------------------------------------
2L File: C:\test\fuyu-no-sanatorium-pb4.wav
2L Rating: 3.0
2L Comment: 
---------------------------------------
ABX Results:
C:\test\fuyu-no-sanatorium-1.1.wav vs C:\test\fuyu-no-sanatorium-pb4.wav
    7 out of 8, pval = 0.035


My impression is that it's safe to use aoTuV pb4 for the upcoming listening test because: I didn't notice any severe regression (although 1.1 is slightly better than pb4 for Mint sample) during the quick evaluation; the improvement for "Genjitsu ni oite" sample is quite positive; and Aoyumi himself, who conducted the continuous effort for the low-bitrate range, recommends pb4 over beta 3 according to his note.