Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Expert Listeners? (Read 12325 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Expert Listeners?

I've seen discussions about various studies that refer to "expert listeners" who can distinguish 320 Kbps audio from lossless. What characterizes these listeners? For example, are they able to pick out a 30 second clip at random from a collection of classical music and ABX it 10/10 times without breaking a sweat? How do they describe the differences they perceive? Do they use flowery audiophile-speak or what?

Expert Listeners?

Reply #1
Could you please provide a reference to the studies you read so we can understand their context?

I can say that when it comes to evaluating lossy audio encoders the terminology here at HA often includes terms like 'pre echo' and 'smearing'. But it is not like wine, chocolate, coffee tasting where they use a standardized flavor wheel with acceptable words, etc. If you look through some of the older listening test threads you'll see some recurring terms though.

Did you read the wiki yet? It has some links to further reading on ABX, etc.

Expert Listeners?

Reply #2
For example, are they able to pick out a 30 second clip at random from a collection of classical music and ABX it 10/10 times without breaking a sweat?


Not 10/10, that's for sure. I would suppose it is closer to "rejecting the null hypothesis of uniform drawing".

 

Expert Listeners?

Reply #3
Since artifacts & compression limitations depend on program material,  I'm not sure if anyone can ABX random samples 100% of the time.  Most people who "bash" MP3 have never done any actual blind ABX tests...  There is a LOT of nonsense in the audiophile community.  Many in the audiophile community don't believe in blind testing!!!!!

Take a look at this Artifact Training Page[/u].  But, be careful....  It might be better if you can enjoy your MP3s without knowing what to listen for! 

I'm sure there's a bit of natural ability as well as a bit of training & experience.    Some people can hear details that others can't...  Some people have better hearing than others...  A very few musicians have perfect pitch, and that's something that can't be learned if you don't possess the natural ability.

Expert Listeners?

Reply #4
I think the chances are more likely with random sample of heavy metal for an individual with acute hearing than a random sample of classical music as it typically demands higher bitrates when using VBR and because the music is more homogeneous throughout.

Expert Listeners?

Reply #5
Since artifacts & compression limitations depend on program material,  I'm not sure if anyone can ABX random samples 100% of the time.  Most people who "bash" MP3 have never done any actual blind ABX tests...  There is a LOT of nonsense in the audiophile community.  Many in the audiophile community don't believe in blind testing!!!!!


Well, the motivation for my post was someone answering the "ABX challenge" in a more or less audiophile forum by presenting perfect foobar2000 results for a (claimed) randomly chosen orchestral snippet encoded at 320 Kbps (LAME 3.97, wouldn't recode with 3.98.4), which to him was easy and perfectly ordinary. I've also encountered people who claim to be able to easily distinguish 128, 160, 192, 256, and 320, from lossless and each other. Then there are those who love to say that lossy isn't even good enough for their car systems. If they provide foobar2000 results, I take them at face value, as I can't set up and monitor the tests myself, but I still can't help but wonder if "expert listeners" in controlled, monitored tests are able to routinely pull off these feats with arbitrary, ordinary music, as opposed to "killer samples". I know I can't, and I'm just curious if these claimed capabilities are real, and how common they are.

Expert Listeners?

Reply #6
It is about knowing what to listen for and experience.  In a typical 3 minute song there's probably only 2 or 3 small artifacts at 320 kbps.  So, no, random 30 second samples wouldn't work,  even for "expert listeners". 

For the tos #8 hounders, search my post history for a successful 320kbps ABX on classical music. 

Expert Listeners?

Reply #7
I searched and saw your one successful ABX attempt.  In one post about it you say it's the exception.  You're telling us now that it is perfectly usual for you with any track?

Expert Listeners?

Reply #8
No.  I'm saying that you have to try really hard and there's only a few isolated artifacts and they require a lot of attention to locate.  It is impossible to be given a clip and say "oh that's 256" because its not a gradual degradation,  its a change in probability of artifacts (at least at higher bitrates).  I know you know this,  greynol,  just clarifying for the OP. 

However, if/when an artifact is identified, its often easy to ABX the track.  in my experience, ABX is an all or nothing deal.

Expert Listeners?

Reply #9
No.  I'm saying that you have to try really hard and there's only a few isolated artifacts and they require a lot of attention to locate.  It is impossible to be given a clip and say "oh that's 256" because its not a gradual degradation,  its a change in probability of artifacts (at least at higher bitrates).  I know you know this,  greynol,  just clarifying for the OP.


FWIW, also my experience with ABXing MP3s.

Quote
However, if/when an artifact is identified, its often easy to ABX the track.


Also my experience.

Quote
in my experience, ABX is an all or nothing deal.


That's what we designed ABX to be.

However, we often end up with mixed bags and arguable results when ABXing audio hardware.  ABX's sharp edge has been mostly advantageous because reality has generally been muddy.

Expert Listeners?

Reply #10
Quote
ABX is an all or nothing deal.

So, ABX result 16/20 is less probable than 19/20 or 20/20?

added: by "less probable" I mean "less likely to obtain".

Expert Listeners?

Reply #11
A very few musicians have perfect pitch, and that's something that can't be learned if you don't possess the natural ability.


Just as a matter of interest, while it was once believed that perfect pitch was inherited, it's now known that it isn't. An adult can acquire it with about a year's systematic practice. Kids acquire it if they learn to associate the names of notes with the pitches they represent before IIRC the age of five.

A good example of how people can become more expert listeners with practice, and it raises some interesting questions about whether some people can hear distinctions that are considered inaudible on the basis of psychoacoustic testing. I'm not aware of any research that shows they can, but there are studies that show that some people are more adept than others at consistently evaluating speakers in blind tests, and that training can improve this skill.

Expert Listeners?

Reply #12
First post here: Please explain how ABX testing relates to expert listeners as opposed to average listeners.

It seems to me that testing for the ability to hear differences in types or equipment or formats, or even testing to hear if something is "audible" is not the same as testing, let's say, in a drug trial.
In simple terms, in a drug trial you are testing for efficacy significantly above a certain threshold (something well above a placebo effect); otherwise the use of the drug can't be seen as useful.

But in listening tests, unless you are trying to determine what the "average" human is capable of, isn't there a situation where a small subset of listeners can "hear" something that typical users can't? I have a musician friend who clearly perceives aspects of classical recordings that I'm not sensitive to. So, in other words, isn't it possible for example, for ABX testing to show that there is no statistically significant audible difference between 2  pieces of equipment or 2 differently formatted recordings, yet there still exist a small number of listeners who can differentiate between them?

Thanks. Sorry for the newbie question.

Expert Listeners?

Reply #13
Quote
isn't there a situation where a small subset of listeners can "hear" something that typical users can't?
Quote
isn't it possible for example, for ABX testing to show that there is no statistically significant audible difference between 2  pieces of equipment or 2 differently formatted recordings, yet there still exist a small number of listeners who can differentiate between them?
Of course!

That’s why, in any experimental design, one must try to use a large and representative/unbiased sample (listeners) when aiming to draw general conclusions.

In most such cases, such an ABX test would be approximating the ‘average’ listener; and for general purposes, isn’t that good? Audio technology for general consumption is targeting exactly them, after all.

If the sample is not sufficiently large or is biased, the test cannot be taken as indicative of any sampling unit (listener) beyond the one/few it was applied to. That’s not necessarily bad: If you conducted an ABX test and found that you had slightly better or worse hearing than average and could/had to adjust your encoding settings accordingly, that makes sense—for you. Equally, in designing a codec for ‘expert’ listeners, the creator would have to preferentially test that subset of people.

TL;DR: In any experiment, use sampling units representative of the population you want your results to apply to.

Expert Listeners?

Reply #14
Results from large "average listener" sets cannot be extrapolated with high certainty to the capabilities of possible expert listeners. The nice thing is, any person claiming to belong to the latter group can be challenged to conduct an ABX test himself. If the outcome is positive there is a high certainty that he does, if not otherwise.

Expert Listeners?

Reply #15
First post here: Please explain how ABX testing relates to expert listeners as opposed to average listeners.


Is that really a question? You seem to be offering a declaration of your answer to it below:

Quote
It seems to me that testing for the ability to hear differences in types or equipment or formats, or even testing to hear if something is "audible" is not the same as testing, let's say, in a drug trial.


Drug testing is usually thought to be related to a passive activity, while participation in a listening test is an activity that presumes active engagement. So, yes they are different.

Quote
In simple terms, in a drug trial you are testing for efficacy significantly above a certain threshold (something well above a placebo effect); otherwise the use of the drug can't be seen as useful.


In a listening test we are in a likewise manner testing for audibility that is significantly above a certain threshold.

Quote
But in listening tests, unless you are trying to determine what the "average" human is capable of, isn't there a situation where a small subset of listeners can "hear" something that typical users can't?


IME listening tests by definition involve people who at least think that they are capable of hearing things that casual listeners don't necessarily notice.  Inevitably, listening tests involve  people who think that they can hear differences that most people don't bother with.  This is a point where the comparisons with drug testing break down quite badly.  People aren't interested in medicines that require special perceptual skills to obtain a benefit from. They want to receive the treatment and have the healing commence forthwith.

Quote
I have a musician friend who clearly perceives aspects of classical recordings that I'm not sensitive to.


You are mixing apples and oranges. Musical differences and sound quality  differences are *not* the same. The parts of the brain that are involved with each are different. Musicians even have a fairly bad reputation for hearing audible differences due to technical changes because they tend to get distracted from technical differences  by the musical issues.

Quote
So, in other words, isn't it possible for example, for ABX testing to show that there is no statistically significant audible difference between 2  pieces of equipment or 2 differently formatted recordings, yet there still exist a small number of listeners who can differentiate between them?


All things are possible, but not all observations are helpful.

Sure, its possible for an ABX test involving one group of listeners to produce a null result, when another similar ABX test involving a different group of listeners produces a positive result. 

Given that there is a well-known and highly recommended activity called "listener training", which tends to improve the sensitivity of listeners, we are pretty sure that we can contrive things so that things like this are observed.

One of the goals of setting up a good listening test is to get the most sensitive possible results.

This is just Science. All findings of Science are provisional. They are only relevant until some better and more reliable finding is obtained.

But let's put this into perspective. At this time 100's and even 1,000's or 10,000's  of listening tests have been done. We know how to relate the measured performance of audio products with the reliable findings of the most sensitive listeners we can train up.  Many of the technical differences that some people claim to hear are bullocks in the sense that they are far, far smaller than the differences that even the most highly trained listeners are able to reliably hear.  OTOH, there are differences that are relatively easy for any reasonably well-trained listener to hear.  The trick is knowing which is which. ;-)

Expert Listeners?

Reply #16
Since artifacts & compression limitations depend on program material,  I'm not sure if anyone can ABX random samples 100% of the time.  Most people who "bash" MP3 have never done any actual blind ABX tests...  There is a LOT of nonsense in the audiophile community.  Many in the audiophile community don't believe in blind testing!!!!!


Well, the motivation for my post was someone answering the "ABX challenge" in a more or less audiophile forum by presenting perfect foobar2000 results for a (claimed) randomly chosen orchestral snippet encoded at 320 Kbps (LAME 3.97, wouldn't recode with 3.98.4), which to him was easy and perfectly ordinary.


I'd say that person is mistaken, fibbing, or using some weird software/hardware combo.  Even the few here with hearing (and training) exceptional enough to be able to ABX 320 from lossless, don't claim they'd be able to do it with a randomly-chosen sample of music (and classical is not necessarily the ''hardest' music to encode).


Expert Listeners?

Reply #17
Hi! I've ABXed 320 from lossless repeatedly. The eig, castanets, and fatboy samples have all fallen before my "ABX skills". Universally, all fail due to pre-echo issues. Sharp attacks just aren't sharp enough. It's not obvious at all; if I didn't have lossless to compare against, I probably wouldn't even notice. I might be able to do this with other samples, but... I'd really rather not ruin lossy encoding for myself entirely.

I'd be quite surprised to hear that someone can ABX an arbitrary orchestral sample at 320 vs FLAC on any kind of reliable basis. But... weirder things have happened. /mnt, here on these forums, ABXed a statistically-significant chunk of certain albums, for example.