Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: transcoding / re-encoding blind listening test (Read 4837 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

transcoding / re-encoding blind listening test

Fifteen years ago I made a very short listening test to check how transcoding (or reencoding) performs against proper audio coding (from CD or lossless source).
Here is a new test.

samples
My first idea was to test four groups of 10 samples each, like this recent test. 10 "easy" popular music samples, 10 "easy" classical music samples, 10 various music samples from HA, and 10 samples known as problem samples.
But for now, I have limited the test to the 10 problem samples group (which is the easiest to test). I'm not sure I can get significant results for the other groups. I'll try later, maybe.

bitrate
15 years ago I used MP3 at 128 kbps (ABR) as output. It was a basic setting for portable use. This time I choose OPUS at 64 kbps. I recently compared several formats at 64 kbps and OPUS@64 appeared to sound as good if not better as LAME@V5 (which is MP3 VBR at ~130 kbps) at least with non-classical music samples. Kamedo2 made the same conclusion here. Opus at 64 kbps seems to be a modern alternative to MP3@128 in 2020.

lossy source
For sources input, I choose encodings around ~260 kbps:
  • MP3: LAME 3.100 V0. It's the highest VBR mode available and I prefered it to CBR/ABR 256 kbps
  • MPC: 1.30 stable at -q8 “Braindead”. It's probably a bit higher than 260 kbps but I'm not entirely sure. For classical music bitrate is near 280 kbps and -q7 “insane” is 240 kbps. So I choose the highest one. I doubt it changes anything to final result.
  • AAC: iTunes VBR at -q109. It's supposed to be near 256 kbps on average (I get ~240 kbps with classical only)
  • OPUS: 1.31 --bitrate 256. It's a true VBR mode
  • WAVPACK: WV 5.3 stable, with -hx3. Quality increases with x4…x6 mode but encoding speed is also lower. I choose made a tradeoff between encoding speed and quality

reference
The reference won't be lossless as in usual tests, but OPUS 64 kbps coming from lossless source. I won't rank OPUS@64 quality but the difference between proper OPUS and transcoded OPUS.

RESULTS: 10 “problem samples” group




comments
• No ABX tests here, but I ranked 31 files and did one mistake (linchpin: confusion between REFERENCE and WV5). It's definitely not luck..
• I insist that I compared OPUS@64 from lossless to OPUS@64 from lossy! The reference isn't the lossless or the CD, and it can be very bad (bachpsichord for example is really awful to my ears once converted to Opus 64 kbps and it becomes difficult to hear additional problem when it's so bad).
• MP3 is clearly inferior as lossy source for transcoding (compared to Opus, Apple's AAC and MPC). It's not really a surprise: most samples have strong impulses/transients, and MP3 even at highest bitrate can't always deal with them transparently. All ten samples were ranked < 5.0 and four of them have a difference that start to be a bit annoying (castanets, eig, enola, glockenspiel).
• AAC, MPC, OPUS have very similar performances. Opus@64 encoded from these formats is usually transparent or with very minor differences compared to OPUS@64 encoded from lossless. And each format only has one sample below 4.0 (enola for MPC, eig for AAC, Linchpin for Opus).
• WavPack lossy wasn't fully transparent seven times. Usually it's small coloration, like brighter sound or slight noisy background. But on two samples it's sounds much worse. The Atem_Lied sample is known for being an issue for WV. Glockenspiel was even worse to my taste.

conclusion
— The test is very limited but I would avoid MP3 at high bitrate as format of choice for optimal transcoding: it adds audible smearing even for 64 modern formats.
— 250 kbps is probably not enough for WavPack lossy: overall quality should be very good but there are some (rare I guess) artifacts that could be very obvious. I recall that my setting was also a tradeoff between speed and quality. WV could sound better at the same bitrate with slower settings.
— at 250 kbps other common lossy formats (OPUS, AAC, MPC) seems to be very efficient as source for transcoding. They don't bring additional artifacts; difference seems to be irrelevant on regular playback. Anyway, what transcoding from high bitrate causes is very minimal compared to what OPUS 64 brings. From these 10 problem samples I don't notice any obvious benefit from one format over another. In other words, 250 kbps lossy seems to be a very solid source for transcoding to a portable setting.


Samples:
• tested OPUS samples are joined to this post (see below)
• lossless samples are also available here: https://www.dropbox.com/sh/zjephy3g54j4gur/AACjGhM9tabl26n7s4ihYl2Ra?dl=0

Hardware:
• AKG Q701 headphone
• Asus R900V computer and native audio chipset (no DAC nor AMP).

Re: transcoding / re-encoding blind listening test

Reply #1
Professional and detailed as always. :)
Also very helpful to (partially) solve my dilemma regarding archiving music.
Thanks @guruboolez
It's interesting to see how some codecs improved in 15 years since last transcoding test and Opus transcoding performance (I can't remember if opus was used before in any transcoding tests).
Wavpack is slightly handicapped by this bitrate. It would be interesting to see how Atem lied and Glockenspiel samples improve with Wavpack on higher settings (maybe -b350hx4 or -b450x4).
Great work!
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: transcoding / re-encoding blind listening test

Reply #2
Thanks very much for your test. It's very interesting and useful.

Offtopic: Nice to see "enola", the sample I submitted for one test 15 ? years ago still in use! I am curious now about it...  :)

Re: transcoding / re-encoding blind listening test

Reply #3
@guruboolez
Are you performing these listening tests at normal listening volume or higher volume?
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

 

Re: transcoding / re-encoding blind listening test

Reply #4
My normal listening volume is rather (too) high. Test is also high.
With classical music (~10 dB lower volume than other) it's not a real problem. With other kind of music and sample I have to lower the volume on normal listening playback. But for tests I tend to keep the volume high with these samples.

Re: transcoding / re-encoding blind listening test

Reply #5
I suppose it should be easier to spot artifacts on higher volume.
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: transcoding / re-encoding blind listening test

Reply #6
Professional and detailed as always. :)
Also very helpful to (partially) solve my dilemma regarding archiving music.
Thanks @guruboolez
It's interesting to see how some codecs improved in 15 years since last transcoding test and Opus transcoding performance (I can't remember if opus was used before in any transcoding tests).
Wavpack is slightly handicapped by this bitrate. It would be interesting to see how Atem lied and Glockenspiel samples improve with Wavpack on higher settings (maybe -b350hx4 or -b450x4).
Great work!

I tried Glockenspiel sample with -b250hx3 using PC speakers.  obvious problem around 2sec.
Upping bitrate to 265k helps .Using -hx4 or even -x4 makes big difference even at 250k.
It seems -x3 and -hx3 produces extra noise & lower quality than using just -h.  Maybe I'll send it
to Bryant later. Its a very good sample.

Re: transcoding / re-encoding blind listening test

Reply #7
Thanks @shadowking
It's very helpful.
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: transcoding / re-encoding blind listening test

Reply #8

I tried Glockenspiel sample with -b250hx3 using PC speakers.  obvious problem around 2sec.
Upping bitrate to 265k helps .Using -hx4 or even -x4 makes big difference even at 250k.
It seems -x3 and -hx3 produces extra noise & lower quality than using just -h.  Maybe I'll send it
to Bryant later. Its a very good sample.

Did you sent this to Bryant? If -hx3 produces lower quality than -h, could this be considered as bug in WavPack?
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: transcoding / re-encoding blind listening test

Reply #9
@guruboolez
Do you think that Mpc @Q10 could improve quality of Enola sample?
Also, I see that lowest quality on 3 samples for mpc/aac/opus is 3.5.
How would you describe 3.5 quality? Is it really something to worry about or is it still very subtle difference that could go unnoticed in normal listening situations?
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: transcoding / re-encoding blind listening test

Reply #10
@guruboolez
Do you think that Mpc @Q10 could improve quality of Enola sample?
I can't be sure without testing but I would say yes.

Quote
Also, I see that lowest quality on 3 samples for mpc/aac/opus is 3.5.
How would you describe 3.5 quality? Is it really something to worry about or is it still very subtle difference that could go unnoticed in normal listening situations?
3.0 = slightly annoying & 4.0 = perceptible but not annoying. 3.5 means a bit annoying and could be noticed in usual listening situations.

Re: transcoding / re-encoding blind listening test

Reply #11
Ok. Thanks. I'll try to ABX Enola sample with different mpc settings and see how it goes.
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: transcoding / re-encoding blind listening test

Reply #12
Belated thanks, @guruboolez

Great to see your thorough and realistic tests are back and even better!
Listen to the music, not the media it's on.
União e reconstrução