Skip to main content
Topic: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz] (Read 2225 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

   Hello,

Everyone is invited to participate.  :)

As some of You already know, we will test xHE-AAC encoder named exhale.

The main target of this test is to figure out which sampling rate, 32 kHz or 44.1 kHz,  is the most optimal for exhale, CVBR mode 2 (~80 kbps).

Testing conditions:
Version - exhale v1.0.3-ea74e998 (Netranger's build)
Bitrate - CVBR mode 2 (~80 kbps)

12 Samples which are well known here on Hydrogenaudio:
01 Castanets
02 Fatboy
03 EIG
04 Bachpsichord
05 Enola Gay
06 Trumpet
07 Applaud
08 Velvet
09 Linchipin
10 Spill the blood
11 Female Speech
12 French ad



The package contains:
ABCHR Java application
ABCHR Windows application
Documentation for both applications "ABCHR Java" and  "ABCHR" for Windows
All predecoded .WAV files (original/uncompressed and xHE-AAC files)


The package was done to run all tests without any additional steps (decoding, copy etc).
Just download the package "Public_test_exhale_80kbkps.zip" to start a test.
 More on how test was prepared
Listeners can choose which application to use (ABCHR Java or ABCHR Win) or any other of his/her choice.

Important note
You don't need to test all samples to participate. Even one single result is already very helpful. Of course, the more you test, the better for the final results' significance.

After you finish some samples, save the results and post (or attach) them in this topic.

Thanks Everybody and I hope You'll enjoy and participate in this test. :)

Package: Public_test_exhale_80kbkps.zip

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #1
Thanks for making this comparison easier!

My results:

Code: [Select]

SAMPLE               32 KHz       44 KHz               ABX 32 vs 44              MAIN ISSUE

01 Castanets          2.5          4.0          16 out of 16, pval < 0.001       Pre-echo
02 Fatboy             1.0          1.8            8 out of 8, pval = 0.004       Noise; grain; uglyness      
03 EIG                2.0          3.5            5 out of 5, pval = 0.031       Irregular smearing  
04 Bachpsichord       3.3          4.0            8 out of 8, pval = 0.004       Smearing or pre-eho
05 Enola Gay          1.5          2.5          12 out of 16, pval = 0.038       Tonal distortion
06 Trumpet            4.5          5.0              no test                      no obvious issue
07 Applaud            3.0          2.0          19 out of 20, pval < 0.001       44K is noisier
08 Velvet             3.8          4.3              no test                      minor lowpass (?)
09 Linchipin          1.7          2.7          12 out of 12, pval < 0.001       excessive smearing
10 Spill the blood    4.2          4.5              no test
11 Female Speech      4.6          4.8              no test
12 French ad          4.4          5.0              no test

                     3.04         3.67    

Except for sample #7 (applauds) the 32 KHz resampled mode 2 is inferior. ABX are not systematic. The most common issue is smearing / preecho / unsharpness / noise increased with 32 KHz (except for sample #7). Tonal issues are also perceptible with Enola Gay sample.

These 12 samples are probably not representative of a generic musical experience but an educated guess would be that 32 KHz resampling would lower the quality of many transients with no obvious benefits (at least for my ears) on other parts.


MINOR ISSUE: the test name for each sample is the same everywhere (Sample01 41_30sec). It's not a problem but I report it.


EDIT: tested with AKG q701 headphone and laptop basic soundcard/headphone output. High Volume.

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #2
@guruboolez

Very nice results. I see You're sensitive to pretty every kind of artifacts: pre-echo, tones etc.  Pretty good listener I would say.   :P
I'll do my part this weekend.

Quote
MINOR ISSUE: the test name for each sample is the same everywhere (Sample01 41_30sec). It's not a problem but I report it.
I've noticed that too but the package was already prepared at that point. Next time I will pay more attention.
Thank You.

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #3
Guys,

You shouldn't be a golden ear nor have expensive audio equipment.  Every participant is valuable.
Even if You can't hear the difference You can still rank some of lossy samples as audibly transparent.

We all have different hearing, preferences, equipment and other ambient conditions.
It's known that an average user has a modest pair of headphones and/or speakers so this test would be representative only if wide range of listeners would participate.

You can post just one single result, later another one or two and so on ...

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #4
Hello,

Able to spot almost... nothing, except Fatboy and (barely) Linchpin.
Mode 2 is definitively my preset.  :)

Thanks for the test!

    AiZ



Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #7
Tested on closed back headphone.
Managed to ABX all samples except 1, 6, 10 and 11, didn't notice a difference.

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #8
Great, thank you all very much! It's interesting how perception differs between the four of you: for some, mainly transient artifacts are audible, for others mainly tonal artifacts while transients sounds good. And for some it all sounds great, which is all a codec developer wishes for ;)

But since the scores are so different, any further volunteers will be very helpful in getting a feeling where this encoder stands for a larger group of listeners, not only experts.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #9
The test will be open until May 30




Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #11
Here are a few results.  I found most but not all samples transparent.  I guess this illustrates a point discussed a few months back about very expert listeners selecting into these "hard" listening tests, thus making the results for near-transparent compression appear artificially low.

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #12
Thanks very much, dsimcha! For the record, attached the results of my own test run, which I completed before guruboolez posted his results.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #13
Great, guys.
And good, Chris, for providing the results for the  test of  your own encoder.

Here is my complete results as well. Phew, I thought I was late.
 

 

Re: Public Test - xHE-AAC exhale encoder, CVBR ~80 kbps [32 kHz vs 44.1 kHz]

Reply #14
Thank You, Guruboolez, AiZ, MetaPixel, dsimcha and Chris for your participation!

The test is finished now.

Here is a global result:



The results of participants:


As conclusion, 44.1 kHz sampling rate should be better than 32 kHz for exhale (xHE-AAC encoder) at ~80 kbps (CVBR mode 2).

 
SimplePortal 1.0.0 RC1 © 2008-2020