Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Personal Blind Listening Test: xHE-AAC (Exhale vs Fraunhofer) vs OPUS at 96 kbps (Read 7442 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Personal Blind Listening Test: xHE-AAC (Exhale vs Fraunhofer) vs OPUS at 96 kbps

In this test I tried to compare what should probably be the three best encoders available at ~96 kbps: OPUS, Exhale xHE-AAC, Fraunhofer IIS xHE-AAC.
AAC, Vorbis, MP3 are older technology and I discarded them in this comparison to keep focus on newer competitors.

Several years ago Opus already appeared as the absolute king at this bitrate, outperforming AAC and Ogg Vorbis. More recently xHE-AAC with Exhale got impressive scores at the same bitrate (kamedo2: 4.15; IgorC: 4.37; myself: >4.50 but as high anchor so score may be overrated).

Last but not least, xHE-AAC (=USAC) can also be tested with a second and totally different encoder (Fraunhofer IIS, commercial encoder included in Poikosoft EZ CDA Converter). This leads to obvious questions:
How do both xHE-AAC encoders compare to each other? Is one better than another? Can xHE-AAC dethrone OPUS at this bitrate?

Fortunately all three encoders have comparable VBR presets for 96 kbps. Data are available in my bitrate table based on ~260 hours of music of various genre.

TECHNICAL DATA

    OpusExhale xHE-AACFraunhofer xHE-AAC
VERSION1.311.1.53.55
PRESET--bitrate 96 --vbr3VBR 3
TYPEVBRCVBRVBR
BITRATE MUSIC100.4 kbps98.2 kbps98 kbps
BITRATE AUDIOBOOKS/MOVIES85.9 / 85.8 kbps81.2 / 83.4 kbps96.6 / 87.5 kbps
ENCODING SPEED¹×320×55×247
¹ : speed on my Intel(R) Core(TM) i7-3610QM CPU @ 2.30GHz (4 cores/8 threads); tested three times; best value reported here; based on 16 different files, 5 minutes each. 2…3 minutes pause between each batch encoding.

To ensure a better scoring for these competitors I add two anchors: a low anchor (Opus 24 kbps) and a mid-anchor (Opus 48). No high anchor here, simply because I expect these 96 kbps competitors to be close to transparency.
In my opinion Opus 24 kbps makes a very good anchor: there are many irritating artifacts and the most important is that it’s not muffled/lowpassed (unlike low bitrate AAC/MP3). For that reason the comparison between anchor and competitor is not like apple and orange but much more than a tasty orange and an unripe one. For mid-anchor, I expect OPUS 48 kbps score to be near 3.0 (at least with music).

SAMPLES

60 samples in 6 groups:

• music: billboard 2010-2020 greatest hits. Standard range for each song: 01.00.000 to 01.20.000. 10 samples, best-sellings music from this chart.. Estimated difficulty for encoders: average.
• music: classical: 10 musical samples from a larger collection. Classical music. Criteria selection: favorite moments. Estimated difficulty for encoders: below average.
• music: HA.io: 10 samples submitted by Hydrogenaudio’s members. Many of them were chosen because they revealed some issues with encoders of the past (usually LAME MP3). Estimated difficulty for encoders: above average.
• music: problem: 10 samples shared on HA.io that revealed strong encoding issues with encoders of the past. Mostly issues like pre-echo or additional noise on transients. Estimated difficulty for encoders: hard to very hard
• voice: audiobooks: 12 samples from audiobooks. 6 male, 6 female. 8 different languages. Criteria selection: random or favorite part. Estimated difficulty for encoders: easy to very easy.
• voice-movies: stereo samples of famous soundtrack (dialog, with or without music and/or ambient noise). 48 KHz, 16 and 24 bit. Criteria selection: cult dialog, favorite moment, or random. Estimated difficulty for encoders: easy.

All samples are available here:
https://www.dropbox.com/sh/zjephy3g54j4gur/AACjGhM9tabl26n7s4ihYl2Ra?dl=0

SOFTWARE & HARDWARE: Lossy files including anchors were created with EZ CDA Converter. The resulting 44.1 KHz files were decoded and renamed with foobar2000 and resampled to 48 KHz (to match OPUS internal sampling rate). Sound normalisation is done with ABC/HR internal tool. Encoding offset is removed by foobar2000; double checked with ABC/HR internal tool. Two first seconds of each encoding are usually discarded from evaluation.
An AKG Q701 was plugged on the headphone jack of my computer (I was unable to run my DAC on ABC/HR). Sound volume varies during the test (rarely quiet, usually medium to high volume).
Last but not least, the graphs and stat analysis were created with kamedo2 online tool. I used Affinity for a quick and ugly photomontage (last graph).


RESULTS




RESULTS FOR MUSIC ONLY : 40 samples



RESULTS FOR VOICE ONLY (AUDIOBOOKS & MOVIES) : 12 + 8 samples



RESULTS FOR EACH GROUP OF SAMPLES






TINY CONCLUSION

► Exhale appears as the most enjoyable encoder at 96 kbps for my own taste and for this group of 60 various samples. It's (and by far) the slowest one but it's also the most “robust” one: there are no big failures on 6 differents groups of samples. Performance is the most homogeneous. Six samples (10% of total) were scored below 4.0, five at 4.0. Therefore it's not perfect.

► Fraunhofer's USAC is also good but fails with classical music. I noticed a kind of ringing here which really annoys me. You can check my ABCHR logs for more details. 18 samples (30% of total) are below 4.0 and five scored exactly 4.0. There are some obvious weakness here and I wouldn’t recommend Fraunhofer’s IIS xHE-AAC over Exhale unless encoding speed is your main concern (more than 4 time faster at this preset; be careful, lower preset with SBR enabled are much slower with Fraunhofer—but still faster than Exhale).

► Opus is fine but has some has more issues on the "critical" group of samples (which surprises me because OPUS is usually stellar on sharpness/transients/preecho — but there are concerns on “tonal” samples, which could also explain some disappointment with classical music). More details can be found I guess on my ABC/HR logs. 16 samples were scored below 4.0 and 3 at exactly 4.0. It’s similar to Fraunhofer’s but far not as good as Exhale.


ON REAL LIFE
I finished the first part of the test (40 music samples) 2 weeks ago and checked immediately after the results. After that I decided to convert a part of my classical library with Exhale at ~96 kbps with EZ CDA converter. 15000 tracks from various sources and sample rate were encoded and resampled to 44100 Hz (it was looooong… but at least I didn’t get any errors nor corrupted files I usually got with foobar2000 resampler and exhale.exe). The average bitrate is 98 kbps which is close to what I get on my bitrate table (98 kbps on average and 96 kbps for classical music only). I can play those files on Android with GoneMAD player which is a pleasant audio player.

My feelings after two weeks of listening Exhale at 98 kbps? Are they consistent with the results I get with ABC/HR?
Well, yes and no. Sound quality is usually excellent and I’m often amazed because such quality was never achievable below 100 kbps. But there are still albums that are not so pleasing.Distortion is most often rather small but it still there and audible—and sometimes irritating.

I would therefore say that Exhale at ~96 kbps is not totally as good as Apple’s AAC at 128 kbps which takes 30% more space.

Re: Personal Blind Listening Test: xHE-AAC (Exhale vs Fraunhofer) vs OPUS at 96 kbps

Reply #1 –
Impressive work!

Nice to see that exhale is quickly reaching maturity.

Re: Personal Blind Listening Test: xHE-AAC (Exhale vs Fraunhofer) vs OPUS at 96 kbps

Reply #2 –
Interesting test, Guru.

exhale is superior that Fraunhofer implementation at 96 kbps. Maybe they should hire Chris  :P  :))

Seriously, I think it's a good approach of having two encoders. Exhale is developed for bitrate range 36+ kbps  while Fraunhofer encoder has an edge at 12-32 kbps.
 

Re: Personal Blind Listening Test: xHE-AAC (Exhale vs Fraunhofer) vs OPUS at 96 kbps

Reply #3 –
Wow, that's a lot of 5.0s. Whether or not Opus or FhG can do a better job on some tracks, it's really something to see such high transparency at 98 kbps.