Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Vorbis Listening Test (Read 78028 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Vorbis Listening Test

Reply #125 –
Quote
So, what's the deal? Which codec should I use? It's not very clear from this theme. The freedom of choice is not, what I need. I wanna know, which codec showed best results. Is there such page ready? And is GTune 3 beta 2 + QKTune beta 3.2 EXPERIMENTAL going to be tested?

Based on the Vorbis listening test, aoTuV won at q 4, so at this nominal bitrate, it is expected to give you the best quality that the Vorbis codec has to offer.

The quality of the new aoTuV (beta 2) is not so definite.  Or perhaps I should say, not as definite as the previous aoTuV that was tested by at least 4 people.

Vorbis Listening Test

Reply #126 –
With regards to the situation, I personally would love to see the new aoTuV be used in the multiformat test,  if it can be reliably verified that it performs better than the previous aoTuV we tested.  There is a risk that there is some regression and the last thing we want is for this to appear in the multiformat test.

However, on the other side of the coin, I did accept the point made by Roberto earlier that we shouldn't go about tuning Vorbis continually in order to improve its chances in the listening test up to the last minute.  That would be unfair.  Therefore I took the initiative and froze my version of QKTune beta 3.2 on that day and Nyaochi also did so for MTb2 the next day.  So I already departed from that rule slightly when I agreed to include the experimental version of aoTuV just before the test and I'm not sure if its proper that I bend the rule again after the Vorbis listening test had completed and we'd all agreed to that version of aoTuV.

Not to place the ball in his court deliberately, but it is ultimately for Roberto to decide whether he will accept this new version of aoTuV (if after preliminary testing, it gets the nod from listeners), to be included in his test or continue to use the one we agreed on in this thread.

Vorbis Listening Test

Reply #127 –
Quote
With regards to the situation, I personally would love to see the new aoTuV be used in the multiformat test,  if it can be reliably verified that it performs better than the previous aoTuV we tested.  There is a risk that there is some regression and the last thing we want is for this to appear in the multiformat test.

However, on the other side of the coin, I did accept the point made by Roberto earlier that we shouldn't go about tuning Vorbis continually in order to improve its chances in the listening test up to the last minute.  That would be unfair.  Therefore I took the initiative and froze my version of QKTune beta 3.2 on that day and Nyaochi also did so for MTb2 the next day.  So I already departed from that rule slightly when I agreed to include the experimental version of aoTuV just before the test and I'm not sure if its proper that I bend the rule again after the Vorbis listening test had completed and we'd all agreed to that version of aoTuV.

Not to place the ball in his court deliberately, but it is ultimately for Roberto to decide whether he will accept this new version of aoTuV (if after preliminary testing, it gets the nod from listeners), to be included in his test or continue to use the one we agreed on in this thread.

If the older version of AoTuV is to be used, it should be made available on Aoyumi's page.  Right now, only the beta2 release is available.

ff123

Vorbis Listening Test

Reply #128 –
Quote
Quote
With regards to the situation, I personally would love to see the new aoTuV be used in the multiformat test,  if it can be reliably verified that it performs better than the previous aoTuV we tested.  There is a risk that there is some regression and the last thing we want is for this to appear in the multiformat test.

However, on the other side of the coin, I did accept the point made by Roberto earlier that we shouldn't go about tuning Vorbis continually in order to improve its chances in the listening test up to the last minute.  That would be unfair.  Therefore I took the initiative and froze my version of QKTune beta 3.2 on that day and Nyaochi also did so for MTb2 the next day.  So I already departed from that rule slightly when I agreed to include the experimental version of aoTuV just before the test and I'm not sure if its proper that I bend the rule again after the Vorbis listening test had completed and we'd all agreed to that version of aoTuV.

Not to place the ball in his court deliberately, but it is ultimately for Roberto to decide whether he will accept this new version of aoTuV (if after preliminary testing, it gets the nod from listeners), to be included in his test or continue to use the one we agreed on in this thread.

If the older version of AoTuV is to be used, it should be made available on Aoyumi's page.  Right now, only the beta2 release is available.

ff123


I still have my own compile:

http://www.rarewares.org/quantumknot/oggenc-aotuv.exe

Hopefully it is bug free

Vorbis Listening Test

Reply #129 –
If anyone has some spare time, by all means, do a test comparing this new aoTuV with the old one.  Irrelevant to whether it does or doesn't make it to the multiformat test, we would still like to know whether it is the best. I'm already contemplating using this to rip my audio collection

Vorbis Listening Test

Reply #130 –
Quote
Not to place the ball in his court deliberately, but it is ultimately for Roberto to decide whether he will accept this new version of aoTuV (if after preliminary testing, it gets the nod from listeners), to be included in his test or continue to use the one we agreed on in this thread.

I won't accept anything without some valid backing up. Otherwise, I'll be making a criticism target out of myself. I only accepted featuring auTuV because there was a listening test behind it I could point critics to.

Vorbis Listening Test

Reply #132 –
Quote
It seems that aoTuVb2 is superior to its previous version on the same samples.

It's probably a pretty safe bet that the other 4 people who listened to all 12 samples will also find beta2 to be better on this sample set (except maybe guru whose correlation with the group wasn't very high).

What is the bitrate penalty for this improvement?

ff123

Vorbis Listening Test

Reply #133 –
Surprisingly, seems the b2 tends to produce smaller files.(at least with the 12 samples) Therefore, IMO including the b2 shouldn't be unfair to the 128kbps features tests.

Vorbis Listening Test

Reply #134 –
Excellent. Thank you, harashin and ff123 (and of course Aoyumi).

Sorry if I looked overly paranoid, but I just wanted to remind everybody not to submit the latest and greatest version of a codec without testing. But then again, Roberto would never accept this anyway.

With all this progress on Ogg Vorbis, it's hard to believe that, only a few months ago, we had the infamous "Is Vorbis dead"-thread. 
Over thinking, over analyzing separates the body from the mind.

Vorbis Listening Test

Reply #135 –
Well, if someone else can come with results that corroborate b2 is better than b1, I will include it in my test.

Anyone up to the challenge? I would need these results by monday the latest.

Edit: Thank-you for testing, harashin.

Vorbis Listening Test

Reply #136 –
My listening result: aoTuV beta2 seems to be better than the previous version. Particularly, preecho problem in Blackwater and FloorEssence samples, background noise problem in MidnightVoyage sample and crash cymbal sound in thear1 sample are improved. IMHO, it's OK to use atb2 for 128kbps instead of at20040402 if someone else does not notice severe regression. 

Here's the result.

Vorbis Listening Test

Reply #137 –
is aoTuV beta2 available as oggenc yet?

Vorbis Listening Test

Reply #139 –
thanx harashin 

Vorbis Listening Test

Reply #140 –
OK. Considering harashin's and nyaochi's results, I would reckon aoTuV b2 should be used instead of b1.

Everyone agree with me? Can we get things rolling?

Vorbis Listening Test

Reply #141 –
Quote
OK. Considering harashin's and nyaochi's results, I would reckon aoTuV b2 should be used instead of b1.

Everyone agree with me? Can we get things rolling?

I think so, yes.  Just gotta make sure that version of oggenc doesn't have the localisation difference with the decimal.  If so, I can provide a binary.

Vorbis Listening Test

Reply #142 –
Quote
I think so, yes.  Just gotta make sure that version of oggenc doesn't have the localisation difference with the decimal.  If so, I can provide a binary.

I think I will go with 4 anyway. So, no worries.

But, of course, a version with . or , hard coded would be welcome

Vorbis Listening Test

Reply #143 –
Quote
I think I will go with 4 anyway. So, no worries.

But, of course, a version with . or , hard coded would be welcome

Oh yeah, q 4. Silly me

Well, here is my hard coded version, for anyone interested.  It uses a . (dot)

http://www.rarewares.org/quantumknot/oggenc-aotuvb2.exe

Vorbis Listening Test

Reply #144 –
Thank you for carrying out harashin, nyaochi, and a comparison test.

Locale fix oggenc (.version) of beta2 can be used from a test page.
This outputs the same data as a reference binary.
aoTuV test page

Moreover, on the page of aoTuV, oggenc and dll of the usual version (for win32) by offer of nyaochi are also prepared. 

Vorbis Listening Test

Reply #145 –
Now that we have some sort of scientific proof that aoTuV is the best Vorbis tuning out of the bunch, do you think it is time to make it the recommended encoder or do we need more rigorous testing before we retire GT3b2?

Vorbis Listening Test

Reply #146 –
Quote
do we need more rigorous testing before we retire GT3b2?

I think so. We, however, can recommend aoTuVb2 at -q4 and below at the moment.

Vorbis Listening Test

Reply #147 –
So let's test aoTuV b2 at 180 kbps nominal bitrate (Q6), so we could compare it with Q5 of GTb2!

Vorbis Listening Test

Reply #148 –
Quote
So let's test aoTuV b2 at 180 kbps nominal bitrate (Q6), so we could compare it with Q5 of GTb2!

Comparing two codecs at such high bitrates is pretty difficult.

Vorbis Listening Test

Reply #149 –
But in this specific case, it's maybe easier. I explain.
There are two known problems with vorbis at this bitrate. By "known", I mean generic problems, occuring with a lot of samples :
- noise issue (very small I admit after -q6): present with GT3b2, maybe removed with aoTuV
- pre-echo issues (very high with CVS code): excellent correction with GT3b (but maybe excessive bitrate inflation), aoTuV need to be tested.

By focusing the tests on these two problems (using percussive samples for pre-echo, and quiet samples for the noise problem), I suppose that we could build a serious opinion about respective quality of these two vorbis encoders.

I was far from internet these two weeks, and my latest aoTuV encoder was the prebeta2 release. I've played a bit with this one, mainly at 96-128 kbps. I've also encoded some  samples at -q6, and noticed sometimes additionnal noise, not very hard to ABX. IIRC [I don't have access now on my main computer], on one sample the noise was audible up to -q7. Obviously, the prebeta2 didn't solve the (or maybe "a") noise issue at high bitrate settings.

Testing is possible, and it's worth to try.
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz