Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Generation loss listening tests (Read 8389 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Generation loss listening tests

I keep seeing people complain about online music stores because they offer music in compressed format and therefore those who want to recompress into a different format will suffer additional loss.

I wonder if anyone's ever conducted a listening test to quantify how audible such generation loss is?

Generation loss listening tests

Reply #1 –
sthayashi tried to organize a "transcode test" once, but not enough people participated to produce any meaningful data.
Read about his efforts (and problems) here and here.

The general advise is to prevent transcoding at all cost and only do it if there's absolutely no other option (and you intend to keep a backup of the original files). There will be a big loss in quality and most of the time there are other ways around the restrictions of various DRM methods.

dev0
"To understand me, you'll have to swallow a world." Or maybe your words.

Generation loss listening tests

Reply #2 –
The best way to figure out if transcoding is bad enough not to do is test it yourself.  Yes, as dev0 says, there is a large loss in quality.  Quality loss yes, but noticable, I'm not sure.  Test some transcodes yourself.  My rule of thumb, if you cannot abx the transcoded sample from the sample it was transcoded from, it's ok.  If you can, don't do it.  Plain and simple.  It's (transcoding) is always regarded as one of the biggest no-nos in audio encoding, but if you cannot tell the difference, there shouldn't be anything wrong with doing it.

Generation loss listening tests

Reply #3 –
I don´t know another thread fitting.
i made frienship with a real "Golden Ear" these days...
He is from Avantgarde-Acoustic and is called Jürgen...
I had a recordings i should prepare for him...
From first playing mpc -q5 and mp3 insane he differrenced all!

What to say more? Sad it is!

Even if Garf arises, it was pretty clear, even for me with these Trios!

Some more Edit....
Visit us in good old Germany Mannheiim...
Me and him invite you! Fun to have listening tets!
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!

Generation loss listening tests

Reply #4 –
Quote
From first playing mpc -q5 and mp3 insane he differrenced all!

This could be quite interesting, could you provide a little more information please?
daefeatures.co.uk

Generation loss listening tests

Reply #5 –
Quote
i made frienship with a real "Golden Ear" these days...
He is from Avantgarde-Acoustic and is called Jürgen...
I had a recordings i should prepare for him...
From first playing mpc -q5 and mp3 insane he differrenced all!

What does golden ears really know about lossy artifacts? As if you can listen for casualy and say 'yep thats mpc q5 and thats mp3 insane' no.. not a human.

I want somebody to try the opposite with golden ear audiophiles:

Ask them if they heard of flac or MA. If they say no then tell them they are lossy codecs better than mp3 and you would like to test their hearing. Play the original  then the flac. They WON'T say 'I can't...' instead they will say which one sounds better!... Even better run some kind of mp3 blind test when they think they are testing mp3 against original. Substitute the mp3 with a flac or keep playing the original wav. I bet they still hear differences.

Generation loss listening tests

Reply #6 –
>>and is called Jürgen...
Is this that admin guy from audiocoding ?
Just curious...

Generation loss listening tests

Reply #7 –
Quote
Ask them if they heard of flac or MA. If they say no then tell them they are lossy codecs better than mp3 and you would like to test their hearing. Play the original  then the flac. They WON'T say 'I can't...' instead they will say which one sounds better!... Even better run some kind of mp3 blind test when they think they are testing mp3 against original. Substitute the mp3 with a flac or keep playing the original wav. I bet they still hear differences.

abx is THERE to eleminate the placebo effect, look here for further questions.
if sb claims to still here differences with lossles encoders (which is most likly a false claim), then in the end, the p-value, in this case standing for a certain degree of possibility of guessing, shows you if he's right or wrong. simple as that.
this is no kind of game, but goes a little bit more in the direction of empirical science...

Quote
What does golden ears really know about lossy artifacts? As if you can listen for casualy and say 'yep thats mpc q5 and thats mp3 insane' no.. not a human.
well, a bit of gift is surly involved, but there is also a lot to be archieved via training.
Nothing but a Heartache - Since I found my Baby ;)

Generation loss listening tests

Reply #8 –
about to do an ABX with some familiar problem samples and lame -aps.

just did Fatboy, but realised i could ABX it at 1 generation (my intention was to find a sample i couldn't ABX at --aps, and then watch the pval shrink with each generation, but it seems i underestimated my hearing)

Generation loss listening tests

Reply #9 –
Quote
abx is THERE to eleminate the placebo effect, look here for further questions.
if sb claims to still here differences with lossles encoders (which is most likly a false claim), then in the end, the p-value, in this case standing for a certain degree of possibility of guessing, shows you if he's right or wrong. simple as that.
this is no kind of game, but goes a little bit more in the direction of empirical science...

Yeah sorry I meant abx. Even if such a test took place using foobar abx test, the audiophile would likely hear differences between flac and wav.. because he/she doesn't know that A is B (both lossless) - you tell them one is lossy and that will bring out their placebo big time and they will screw up the test.

Wombat should do a proper abx test on the pc to see if this friend can abx lame -APS. I am also interested to hear about his test.

But I have doubts. Like you said with proper abx training it may be possible to a/b APS on a few samples, however most people that claim to hear everything have no abx training to start with. The worse thing is that they are way more suseptible to placebo than someone with abx experience.

Generation loss listening tests

Reply #10 –
Quote
>>and is called Jürgen...
Is this that admin guy from audiocoding ?
Just curious...

Hans-Jürgen is not Admin at Audiocoding. He's not even moderator. He just hangs there a lot because he dislikes HA and Audiocoding is the only audio forum left for him to advertize Faac.

Generation loss listening tests

Reply #11 –
shadowking:  that's kinda nonsense... random element turns placebo into nothing.  even if i knew there was blade v flac, if i couldn't hear the difference i'd get a high pval (this is hypothetical of course... blade is awful).

here's my result on some problem sample i had lying around.  not that conclusive as i could ABX all of them in less than 16 goes (i stop at pval < 0.01).

Quote
41_30sec.flac

1st generation --aps v flac:

10/11 - extremely difficult (2.3 - 4.7), no difference in sound per se, more a slightly less sharp attack on the hi-hat


2nd gen --aps v flac:

2.3 sec - ~3.5 sec  (may be inaccurate, got sidetracked)
8/8 - surprisingly difficult.  (very) slightly smeared transient


3rd generation --aps v flac:

obvious quality loss... not sure how much point there is in ABX, but nonetheless...
ABX'ed in about 10 seconds 8/8.  the others took considerably longer.


this was generated with foobar's ABX and textpad.  the files were simple transcodes from FLAC with LAME 3.92 at --alt-preset standard, each one being transcoded from the last.

at 1 transcode transparency is only just noticably broken, and at 2 transcodes (why anyone would do THAT is beyond me) there is a very noticable loss in quality.  there are blocking artefacts (ie, the 50% block overlap isn't enough to hide the edges of blocks) and volume pumping, plus a _slight_ warble.  better than i expected really.

Generation loss listening tests

Reply #12 –
Quote
Yeah sorry I meant abx. Even if such a test took place using foobar abx test, the audiophile would likely hear differences between flac and wav.. because he/she doesn't know that A is B (both lossless) - you tell them one is lossy and that will bring out their placebo big time and they will screw up the test.

there is no placebo effect while abxing.
Quote
abx is THERE to eleminate the placebo effect


Quote
shadowking: that's kinda nonsense... random element turns placebo into nothing. even if i knew there was blade v flac, if i couldn't hear the difference i'd get a high pval
exactly.

I suggest you have a bit of a read into this topic.
Nothing but a Heartache - Since I found my Baby ;)

Generation loss listening tests

Reply #13 –
Quote from: Digga,Apr 11 2004, 05:47 AM
Quote from: shadowking,Apr 11 2004, 01:54 P[/QUOTE

there is no placebo effect while abxing.
Quote
abx is THERE to eleminate the placebo effect



Sometimes I imagine differences in abx tests when I  cannot clearly tell A / B apart ,, but at the same time something seems different but not enough - in reality A / B are the same but my ears or mind trick me. This happens when I heard the artifact before and then expect to abx it everytime, but it doesn't always happen esp with faint artifacts. I hear it in my head but don't always abx. Don't know how to explain this further  its like the expectation leads me to mistakes sometimes..

So i guess that is bad judgement or fatigue and not placebo.

Generation loss listening tests

Reply #14 –
I am curious as to why audiophiles hear differences between 128k, 192 ,320 , mpc, ap insane. I think its because they are audiophiles: 128k vs 129k - the difference is minute but its there and they will tell you that they hear a tiny difference in quality.

I read this often on the web, hi-fi publications etc. As long as there is a difference no matter how big or small - they hear it.

Generation loss listening tests

Reply #15 –
Quote
I wonder if anyone's ever conducted a listening test to quantify how audible such generation loss is?

I have tested MPC Q5 transcoded to Lame preset fast medium several times. All i wanted was acceptable playback on the portable discman. I had to make sure
that no beast will come out of my headphones and strangle me for this atrocity. java script:emoticon(':D')

Anyway casual abxing of the original vs transcoded files proved transparent for me at the time. I am sure I could eventually find a flaw but its portable listening after all.

So if you must transcode then i suggest that you try this method. If nothing terrible happens then its ok for you. Just remember not to encourage others with this info and don't share the transcoded files.

Generation loss listening tests

Reply #16 –
this is probably voodoo, but i find that mpc -> lame stands up better than lame -> lame.

it's possibly because transform coefficient quant error is intrinsically different from narrowband quant error, so the result still more q noise, but split between different sounding noises.  i'll have to do some ABXing here, because i'm talking on intuition alone.

my lame test tells me that no matter what you're doing, don't go beyond 1 transcode generation or there'll be poo sounds no matter what (lossy) compression scheme you use.

Generation loss listening tests

Reply #17 –
If you're looking for really good transcoding (i.e. generational) performance, you should check out the lossy quantization codecs... den did extensive listening tests with WavPack lossy, beginning here, in case you're interested.

Generation loss listening tests

Reply #18 –
i wouldn't dare try listening for quality loss in a single transcoding, but it DOES exist. back in the day of FhG's first windows encoder which was quite a deal faster than L3Enc, another encoder came out called MP3 Compressor and was a bit faster (10-30%? don't remember the number), but it turns out it was based on a leaked beta of the FhG encoder and it had inferior quality which was illustrated by doing a WAV > MP3 > WAV and back to MP3 around 10 times total. there was a definite loss in quality anyone could hear (even your grandparent could hear it).

Generation loss listening tests

Reply #19 –
Why is it so obvious that transcoding from lossy to lossy adds notable quality loss?
I mean, psychoacoustic codecs seek for information that is inaudible before removing it. If decoded stream from lossy codec contains only information that is audible, based on what would encoder remove any information?
I imagine that in certain cases it comes down to simply storing decoded stream in new format and only some noise is added.
Of course, I'm not sure of any of it, just thinking aloud.

To get best transcoding, it seems to me that encoder should be set to most extreme quality it can offer, so it removes minimal information. Given that excess information has already been removed, it still should compress quite well, isn't it? Decoded stream isn't carrying same information density as original, so it should pass through psychoacoustic shaping quite intact before getting to the normal bit reduction storing.

I wonder, how well does say decoded mp3 compress with ape or any other lossless audio codec? I guess it should compress much better than original.

In a sense, chain of transcodings limits quality by its weakest element in the chain.

I wonder, if transcoded with standard settings, codec is programmed to remove at least something, and final file is reduced in size together with quality. But what quality loss is expected if file size is same after transcoding and codec is told to be "paranoid" in what it removes? Or, is it expected that file size grows compared to transcoding source in that case?
It really really did sound different. Not in a placebo way.

Generation loss listening tests

Reply #20 –
Quote
I don´t know another thread fitting.
i made frienship with a real "Golden Ear" these days...
He is from Avantgarde-Acoustic and is called Jürgen...
I had a recordings i should prepare for him...
From first playing mpc -q5 and mp3 insane he differrenced all!

What to say more? Sad it is!

Even if Garf arises, it was pretty clear, even for me with these Trios!

Some more Edit....
Visit us in good old Germany Mannheiim...
Me and him invite you! Fun to have listening tets!

Come on Wombat - you know you can't just leave it at that!

You've at least got to give test methodology, equipment, and samples...

Cheers,
David.

Generation loss listening tests

Reply #21 –
Quote
I mean, psychoacoustic codecs seek for information that is inaudible before removing it. If decoded stream from lossy codec contains only information that is audible, based on what would encoder remove any information?


it's all a matter of how the information is removed.  there's no way an encoder can tell what is _already_ quantization noise in the stream.

all a psychoacoustic encoder (well, mp3 in this case) does is split the signal into bands (supposedly aligned to human hearing critical bands) and calculates based on tonality and ATH and some clever maths which bands are less likely to be heard, and therefore less important.

it's not a matter of removing frequencies as such (though effectively it is), but rather storing the data with less precision.  turning 3.141592653589793 into 3.14 for example (yes, that's from memory.. i'm a nutter).

but what happens is that in a transcode, the encoder assumes each band to have no noise already.  it will generate very similar masking thresholds and therefore give the most quantization error to the parts that are _already_ very inaccurate.

this means that if the original encode was only just transparent, then a transcode _cannot_ be transparent, unless it has extra information from the original encoding process.  in the worst case the quantization noise will double per transcode, but in reality it's a little less.


[span style='font-size:8pt;line-height:100%']Edit by Pio2001 : corrected the 3 last digits of Pi[/span]

Generation loss listening tests

Reply #22 –
Quote
it's all a matter of how the information is removed.  there's no way an encoder can tell what is _already_ quantization noise in the stream.

What do you call noise here? I assume not something in analog equipment you'd call white noise, but something lossy codec ppl like to call when they talk about deviation from original. That is not something you'd need to encode, that is something that arises in both codecs. Whether it gets any worse is a question. If you make averaging of 0.7 upto integer, then you can add quantisation noise only once, after that its integer no matter how many times you repeat.

Quote
it's not a matter of removing frequencies as such (though effectively it is), but rather storing the data with less precision.  turning 3.141592653589793 into 3.14 for example (yes, that's from memory.. i'm a nutter).
That might be good example. If previous codec did reduction of precision of pi to 3.14, then what has next codec any more to do with this? 3.14 is already reduced to closest allowable noise floor, for codec this should be incompressible data.
If the signal ever was converted to analog and then again lossy compressed, then we could imagine degradation of compression because 3.14 is not precisely 3.14 anymore. But pure digital transcoding - there is no intrinsic noise added between codings. This means, there is no "less precision" anymore (given of course, similar transparency objectives).

If it was waveform precision, then "stepness" could be perceived as white noise by encoder, but in subband encoders, reduced precision is about spectral components, which are reduced in precision only upto a limit.

Quote
but what happens is that in a transcode, the encoder assumes each band to have no noise already.  it will generate very similar masking thresholds and therefore give the most quantization error to the parts that are _already_ very inaccurate.
By generating very similar masking thresholds second codec does not degrade signal further, but only changes nature of its noise somewhat. Some codec would reduce pi to 3.142, some would reduce to 3.141, some would reduce to 3.14. Thats not something of cumulating nature - it has limit, defined by most agressive codec. Giving 3.14 to codec that would pass 3.140 or 3.141 or 3.142 intact have zero impact on that 3.14

That encoder assumes each band to have no noise already is irrelevant, imo. Encoder has no idea of noise anyway, what it sees is either excessive precision and information, or lack of these.

Quote
this means that if the original encode was only just transparent, then a transcode _cannot_ be transparent, unless it has extra information from the original encoding process.  in the worst case the quantization noise will double per transcode, but in reality it's a little less.
I don't see your conclusion following from the above. I could imagine several cases where this would be true, but I'm not seeing how this could be stated as universally true statement.
My idea was that in best case the quantization noise will not change at all, and thus transparency will remain intact. This must depend on codec pairs used in transcoding. I can't see anything given here. Where is my error?
It really really did sound different. Not in a placebo way.

Generation loss listening tests

Reply #23 –
Quote
this is probably voodoo, but i find that mpc -> lame stands up better than lame -> lame.

I've noticed this too, in my tests. Lame aps->lame 128 just sounds worse than mpc q5->lame 128.

I did several ABX tests a couple years ago, comparing lame aps and insane encoded from either the original or transcoded from mpc q5. I only tested pre-echo samples (fatboy, castanets, drone_short), because back then, those were the only samples I could ABX with lame. Basically, if I could pick out mpc -> aps or mpc -> insane, I could do the same if I encoded from the original. The transcoded versions never sounded worse or better to my ears.

I think it was Gecko who once said that, in his tests, mpc q5 -> lame aps would sometimes generate little pops here and there, and that starting with mpc q6 would eliminate most, but not all, of the pops. I've been listening to transcoded q5 mpc -> lame aps on my portable player for years, and I haven't heard any conspicuous pops or artifacts that made me want to go home and try to ABX the transcodes.

Generation loss listening tests

Reply #24 –
Quote
Quote
it's not a matter of removing frequencies as such (though effectively it is), but rather storing the data with less precision.  turning 3.141592653589793 into 3.14 for example (yes, that's from memory.. i'm a nutter).
That might be good example. If previous codec did reduction of precision of pi to 3.14, then what has next codec any more to do with this? 3.14 is already reduced to closest allowable noise floor, for codec this should be incompressible data.
If the signal ever was converted to analog and then again lossy compressed, then we could imagine degradation of compression because 3.14 is not precisely 3.14 anymore.

I'll try to explain using an imaginary codec.

The encoder doesn't get just one number but a whole stream of numbers. This isn't limited to the imaginary codec; it applies to most real world codecs also. Please assume that the following numbers represent audio data:
1.7 2.4 3.2 4.4 5.1

Our encoder is smart. Depending on the input (in our case the input is a bunch of numbers, not just a single number) it can either leave a number alone or it can round it to the nearest integer. What the encoder chooses to do is based on it's psychoacoustic model which not only considers one single number but also the numbers in the vicinity. Assume the imaginary psychoacoustic model specifies the following: a number can be rounded to the nearest integer, if it's successor is either larger or smaller by 1.

In pseudo code (a is the input, b the output):
if abs( a - a[i+1] ) > 1
  then b = round(a)

For example:
|1.7 - 2.4| = 0.7
Thus, 1.7 is rounded to 2.0.

After the first pass, the signal looks like this: (the last number is unchanged as it has no successor)
2.0 2.0 3.2 4.0 5.1

If you run it through that algorithm a second time you will see that according to the rules, 3.2 may be rounded too, while in the first pass it was forbidden. The results is:
2.0 2.0 3.0 4.0 5.1
As you can see, there would be no further generation loss.

This is of course a strong simplification of matters, but the principles should be the same.