Low Bitrate Listening Test, 32kbps, HE-AAC vs LC-AAC vs Opus (HowTo?)

2017-07-07 21:56:08

What's the correct way to test very low bitrates (for the purpose of low quality music streaming), When it's very easy to ABX the encodings being considered.

I want to compare:
* Opus @ 32kbps and 40kbps
* HE-AAC @ 32kbps and 40kbps
* LC-AAC @ 64kbps and 80kbps

Now I have seen a few times in this forum that HE 48kbps ~  LC 64kbps ~ MP3 96kbps . But those posts are old. I think HE has come a long way since then because I find that HE 32kbps is close to Opus 32kbps (opus still sounding better) and WAY better than LC 64kbps.

In Summary LC 64kbps sounds low-passed and is missing a lot of the high frequencies. It's the easiest to differ from the original FLAC. But all the other 32kbps samples are quite easy to differ with ABX tests. LC AAC gets noticeably better from 80kbps and above. Which matches the typical recommendation of switching to HE from 80kbps and lower.

Just to be clear I'm aware of ABX tests, but I find that particular tests are not ideal for such low bitrates. And I don't know if there is another test that would be more appropriate for such samples.

Re: Low Bitrate Listening Test, 32kbps, HE-AAC vs LC-AAC vs Opus (HowTo?)

Reply #1 – 2017-07-07 22:15:24

http://wiki.hydrogenaud.io/index.php?title=ABC/HR ?

Re: Low Bitrate Listening Test, 32kbps, HE-AAC vs LC-AAC vs Opus (HowTo?)

Reply #2 – 2017-07-10 23:06:27

If I understand what you're trying to do, you can just rank them by quality/preference and decide which is worst and which is best.     Or, you could get a panel of listeners and find the consensus.

...Sometimes people try to say the vinyl records are the equivalent of 11 or 12 bits (on a "good day", I'd guess).  But, degraded digital never "sounds like" vinyl so it's a matter of deciding which one you prefer.   Or, maybe you find the point where 50 listeners prefer the vinyl and 50 listeners prefer the digital and declare them "equal" at that resolution.

Same thing with video.    Low-bitrate or low-resolution digital video never "looks like" VHS, but presumably you could find a point where half of the viewers prefer one and half prefer the other.

...And, assuming you can really hear a difference in a blind listening test, there's no point in making the preference test blind.   If you want to find out if people prefer the taste or Coke Pepsi, you do that blind.    But if you want to find out if people prefer the taste of orange soda or grape soda, there's no point in doing a blind test.  

Re: Low Bitrate Listening Test, 32kbps, HE-AAC vs LC-AAC vs Opus (HowTo?)

Reply #3 – 2017-07-11 06:03:03

DVDdoug, your suggestions are not going to help arrive at a reliable result.  The test should still be blind and be done with a thought-out methodology. This isn't an orange vs grape situation; just because the encoded samples can be easily ABXed from the original doesn't mean that for every single sample a listener will know immediately which encoder and bitrate was used.

MUSHRA and ABC/HR are two of the more common rating methodologies. ABC/HR is recommended for things that are close to transparency; the HR is for hidden reference, which gives a high anchor. When it's patently obvious to everyone which is the original, that's not really a benefit and MUSHRA will be as good.

Re: Low Bitrate Listening Test, 32kbps, HE-AAC vs LC-AAC vs Opus (HowTo?)

Reply #4 – 2017-07-11 06:31:20

It's not meant to be an extensive test. This test is for myself, so that I can avoid placebo effects.

So blind tests can be useful. I decided the way to go about it is to make the samples shorter than I'm used to. So instead of testing full songs, I'll try splitting said song into shorter clips of just a few seconds. I wouldn't be able to wait for key parts of the song, so maybe then, some of the samples can be transparent even though I'm using such low bit rates.

I'm used to using foobar2000, and my first search didn't find any abc/hr for it. (but I didn't search for too long)