Hello,
Everyone is invited to participate. :)
As some of You already know, we will test xHE-AAC encoder named exhale. (https://hydrogenaud.io/index.php?topic=118888.0)
The main target of this test is to figure out which sampling rate, 32 kHz or 44.1 kHz, is the most optimal for exhale, CVBR mode 2 (~80 kbps).
Testing conditions:
Version - exhale v1.0.3-ea74e998 (Netranger's build) (https://hydrogenaud.io/index.php?topic=118888.msg983075#msg983075)
Bitrate - CVBR mode 2 (~80 kbps)
12 Samples which are well known here on Hydrogenaudio:
01 Castanets
02 Fatboy
03 EIG
04 Bachpsichord
05 Enola Gay
06 Trumpet
07 Applaud
08 Velvet
09 Linchipin
10 Spill the blood
11 Female Speech
12 French ad
The package contains:
ABCHR Java application
ABCHR Windows application
Documentation for both applications "ABCHR Java" and "ABCHR" for Windows
All predecoded .WAV files (original/uncompressed and xHE-AAC files)
The package was done to run all tests without any additional steps (decoding, copy etc).
Just download the package "Public_test_exhale_80kbkps.zip" to start a test.
More on how test was prepared (https://hydrogenaud.io/index.php?topic=118888.msg983108#msg983108)
Listeners can choose which application to use (ABCHR Java or ABCHR Win) or any other of his/her choice.
Important note
You don't need to test all samples to participate. Even one single result is already very helpful. Of course, the more you test, the better for the final results' significance.
After you finish some samples, save the results and post (or attach) them in this topic.
Thanks Everybody and I hope You'll enjoy and participate in this test. :)
Package: Public_test_exhale_80kbkps.zip (https://drive.google.com/file/d/1Cm0E6tt0gG0OT_zjzSqwBpfT3-nrPspp/view?usp=sharing)
Thanks for making this comparison easier!
My results:
SAMPLE 32 KHz 44 KHz ABX 32 vs 44 MAIN ISSUE
01 Castanets 2.5 4.0 16 out of 16, pval < 0.001 Pre-echo
02 Fatboy 1.0 1.8 8 out of 8, pval = 0.004 Noise; grain; uglyness
03 EIG 2.0 3.5 5 out of 5, pval = 0.031 Irregular smearing
04 Bachpsichord 3.3 4.0 8 out of 8, pval = 0.004 Smearing or pre-eho
05 Enola Gay 1.5 2.5 12 out of 16, pval = 0.038 Tonal distortion
06 Trumpet 4.5 5.0 no test no obvious issue
07 Applaud 3.0 2.0 19 out of 20, pval < 0.001 44K is noisier
08 Velvet 3.8 4.3 no test minor lowpass (?)
09 Linchipin 1.7 2.7 12 out of 12, pval < 0.001 excessive smearing
10 Spill the blood 4.2 4.5 no test
11 Female Speech 4.6 4.8 no test
12 French ad 4.4 5.0 no test
3.04 3.67
Except for sample #7 (applauds) the 32 KHz resampled mode 2 is inferior. ABX are not systematic. The most common issue is smearing / preecho / unsharpness / noise increased with 32 KHz (except for sample #7). Tonal issues are also perceptible with Enola Gay sample.
These 12 samples are probably not representative of a generic musical experience but an educated guess would be that 32 KHz resampling would lower the quality of many transients with no obvious benefits (at least for my ears) on other parts.
MINOR ISSUE: the test name for each sample is the same everywhere (
Sample01 41_30sec). It's not a problem but I report it.
EDIT: tested with AKG q701 headphone and laptop basic soundcard/headphone output. High Volume.
@guruboolez Very nice results. I see You're sensitive to pretty every kind of artifacts: pre-echo, tones etc. Pretty good listener I would say. :P
I'll do my part this weekend.
MINOR ISSUE: the test name for each sample is the same everywhere (Sample01 41_30sec). It's not a problem but I report it.
I've noticed that too but the package was already prepared at that point. Next time I will pay more attention.
Thank You.
Guys,
You shouldn't be a golden ear nor have expensive audio equipment. Every participant is valuable.
Even if You can't hear the difference You can still rank some of lossy samples as audibly transparent.
We all have different hearing, preferences, equipment and other ambient conditions.
It's known that an average user has a modest pair of headphones and/or speakers so this test would be representative only if wide range of listeners would participate.
You can post just one single result, later another one or two and so on ...
Hello,
Able to spot almost... nothing, except Fatboy and (barely) Linchpin.
Mode 2 is definitively my preset. :)
Thanks for the test!
AiZ
I'm lazy today so just two results as for now. Will do more later.
More results
Tested on closed back headphone.
Managed to ABX all samples except 1, 6, 10 and 11, didn't notice a difference.
Great, thank you all very much! It's interesting how perception differs between the four of you: for some, mainly transient artifacts are audible, for others mainly tonal artifacts while transients sounds good. And for some it all sounds great, which is all a codec developer wishes for ;)
But since the scores are so different, any further volunteers will be very helpful in getting a feeling where this encoder stands for a larger group of listeners, not only experts.
Chris
The test will be open until May 30
More results
Here are a few results. I found most but not all samples transparent. I guess this illustrates a point discussed a few months back (https://hydrogenaud.io/index.php?topic=118446.0) about very expert listeners selecting into these "hard" listening tests, thus making the results for near-transparent compression appear artificially low.
Thanks very much, dsimcha! For the record, attached the results of my own test run, which I completed before guruboolez posted his results.
Chris
Great, guys.
And good, Chris, for providing the results for the test of your own encoder.
Here is my complete results as well. Phew, I thought I was late.
Thank You, Guruboolez, AiZ, MetaPixel, dsimcha and Chris for your participation!
The test is finished now.
Here is a global result:
(https://i.ibb.co/L9p4ZzT/exhale-32k-Hz-vs-44k-Hz-80kbps-CVBR.png)
The results of participants:
(https://i.ibb.co/xj9YQpg/listener-results.png)
As conclusion, 44.1 kHz sampling rate should be better than 32 kHz for exhale (xHE-AAC encoder) at ~80 kbps (CVBR mode 2).