Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: What we measure is what we hear (Read 24732 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

What we measure is what we hear

Reply #25
Subjective impressions are, after all, impressions relative to the subject, most generally the particular subject only, not relative to the data or the general population.

They might tell us something about the subject, but not much about the object. In some case, subjects even hear what they expect to hear when there is no signal (data/sound) at all. That alone implies the possibility that what they report hearing in the vague terms used in such reports do not relate to anything at all coming from the traducer (speaker/headphone/etc.)

What we measure is what we hear

Reply #26
Subjective impressions are, after all, impressions relative to the subject, most generally the particular subject only, not relative to the data or the general population.

They might tell us something about the subject, but not much about the object. In some case, subjects even hear what they expect to hear when there is no signal (data/sound) at all. That alone implies the possibility that what they report hearing in the vague terms used in such reports do not relate to anything at all coming from the traducer (speaker/headphone/etc.)


A properly designed and operated subjective test IS a form of measurement.

Audiophile tests generally (almost always) need not apply, of course.

But, a subjective test, run as a DBT, using a proper design, with anchors and controls, is a measurement.  A measurement that is a true pain in the behind to do, perhaps, but it IS a measurement, and it most often even gives a useful noise level for the measurement.
-----
J. D. (jj) Johnston

What we measure is what we hear

Reply #27
TL;DR:
MUSHRA and other ABC-style tests are subjective tests.  This is because the person taking the test assigns a score to each stimuli in order to declare a personal preference.

What we measure is what we hear

Reply #28
So far the best measuring headphones I have tried sound the best and the best measuring amps sound the best so there is no contradiction between what sounds good and what measures well to me. It makes it easy.
FLAC -> JDS Labs ODAC/O2 -> Sennheiser HD 650 (equalized)

What we measure is what we hear

Reply #29
What's the "best measuring" frequency response of a headphone? I wonder...

What we measure is what we hear

Reply #30
So far the best measuring headphones I have tried sound the best and the best measuring amps sound the best so there is no contradiction between what sounds good and what measures well to me. It makes it easy.

AKA "expectation bias"... it rarely misses a shot!

P.S. What you said is not wrong in principle, but fact is that human ear is not linear neither has infinite precision...
... I live by long distance.

What we measure is what we hear

Reply #31
What's the "best measuring" frequency response of a headphone? I wonder...


The one which best matches your ear shape, of course!


Creature of habit.

What we measure is what we hear

Reply #32
A properly designed and operated subjective test IS a form of measurement.
I thought the food industry is doing this all the time:
http://www.ift.org/
Quote
Sensory evaluation has been defined as "a scientific discipline used to evoke, measure, analyze, and interpret reactions to those characteristics of foods and materials as they are perceived by the senses of sight, smell, taste, touch, and hearing" (IFT, 1975).

Perhaps we should be less scared to be subjective. All humans "suffer" from it. The real art is in finding out which variables are responsible.

Happy new (multisensory) year !

What we measure is what we hear

Reply #33
I don't think this forum is scared of subjective opinions; we simply believe that expectation bias should not have a role at the table when it comes to gauging and expressing them.

What we measure is what we hear

Reply #34
Many people have worked on measuring what you hear. No perceptual coder (MP3, AAC, AC3, etc) would work if many people had not only done that, but also done a good job of it.

My own experience makes me wonder how well human hearing has been measured. I'm sure average human hearing has been well-studied, but how many outliers are there in the world population? My daughter, for example, can hear the high frequency tones used by cat-repelling devices. She's demonstrated this many times, but my wife and I can't hear them at all (nor can most people).


Since you haven't given the actual operating frequency of this device, your claim lacks global meaning.

It is well known that children generally have better and more extended HF hearing than adults. Human ears have geometry that determines how they work and when they grow, the geometry shifts towards lower frequencies.

Quote
I wonder if the hearing range of various human populations has been well-studied (such as African tribes with certain members who can hear low frequency tones emitted by elephants).


We don't need to jump down every rabbit hole to know where to find rabbits every time we need to. ;-)

Perhaps one of the most cogent comments on this thread is from JJ, who makes the point that since lossy coders provide substantial amounts of data reduction with minimal sound quality loss all things considered, and they work based on what we know about human hearing, we must know quite a bit of useful information about human hearing.

Quote
Is it fair to say that this good work has been done primarily based on averages of 1st World populations?


More to the point there has been no need to develop special lossy encoders for every country, continent or people group. This is particularly true when good unbiased listening tests are used to determine what people are and are not hearing.

What we measure is what we hear

Reply #35
Subjectivity can still provide valuable data points (especially negative impressions).

The problem with subjective non-blind tests is that perception varies from moment to moment. Something that sounds bad now might sound excellent tomorrow or even five minutes later. This is the main reason most subjective audiophile reviews are useless, except when something is extremely bad.

--Ethan
I believe in Truth, Justice, and the Scientific Method

What we measure is what we hear

Reply #36
Subjectivity can still provide valuable data points (especially negative impressions).


The above is like saying that radioactivity can lead to thermonuclear explosions.

Subjectivity can provide valuable data points, but the frequency of incidence of this happening can be disappointingly low.

Consider all of the high end audiophile publications. Thousands or tens of thousands of pages of subjective impressions per week or month that would simply cease to exist if proper listening test protocols were used.

What we measure is what we hear

Reply #37
Subjectivity can still provide valuable data points (especially negative impressions).

The problem with subjective non-blind tests is that perception varies from moment to moment. Something that sounds bad now might sound excellent tomorrow or even five minutes later. This is the main reason most subjective audiophile reviews are useless, except when something is extremely bad.

--Ethan

This could be true to a certain limited extent, but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.
What can really change and by large amounts are expectations in sighted tests, when the tester knows what he's listening to and could link to the listened device side effects of other precedent or contemporary experiences, different knowledge, true or supposed, even the mood of the moment...
... I live by long distance.

What we measure is what we hear

Reply #38
Maybe we should take a close look at the meaning of the word perception. From where I stand, and from having an opinion shaped by what I have read by many of the fine members here and elsewhere with demonstrated expertise in the area, I will add that expectation bias does in fact alter our the perception of what we hear.  The suggestion of a difference in X (breath/openness/dancability/PRAT but also non-BS qualities) can result in the actual and very real perception of that difference in X to the individual, even when the stimulus is held constant.

So no, changing perception does not have to affect DBT results necessarily.  In the case of having placebophiles with strong expectation bias we often see the reverse: DBT affects the perception, or maybe not until the results of the test are revealed. That's still no guarantee since some people think that getting 13 right out of 20 means they passed the test.  Still others will continue to cling to their faith in fairy dust, especially when the perceived differences once again return when the constraint of a DBT has been lifted.

What we measure is what we hear

Reply #39
Next, on to wine tasting.... 

What we measure is what we hear

Reply #40
Feel free to start a new topic, replacing the word hear in the subject line with the word taste.  Be sure to post it in the off-topic forum.

What we measure is what we hear

Reply #41
I don't think this forum is scared of subjective opinions; we simply believe that expectation bias should not have a role at the table when it comes to gauging and expressing them.



Amen to that. Sorry, but it had to be said twice.
-----
J. D. (jj) Johnston

What we measure is what we hear

Reply #42
What's the "best measuring" frequency response of a headphone? I wonder...


On whose head, with what haircut, under what circumstances, with what ear canal volume and shape?

Martel has a real point here.
-----
J. D. (jj) Johnston

What we measure is what we hear

Reply #43
This could be true to a certain limited extent, but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.


To some extent that's true.

Even something so simple as measuring one's hearing can be very different on day 1 vs. day 3 of getting off of the plane, or if you spent 8 hours on the show floor vs. ran through it with earplugs.

But this kind of thing can be managed, and that's also what controls are for.
-----
J. D. (jj) Johnston

What we measure is what we hear

Reply #44
Feel free to start a new topic, replacing the word hear in the subject line with the word taste.  Be sure to post it in the off-topic forum.


Actually, I know of at least 1 or 2 good books on sensory evaluation of wine that directly read on audio testing if you think even a little bit.
-----
J. D. (jj) Johnston

What we measure is what we hear

Reply #45
You're right, I just don't know that we should drop comparing/correlating audio measurements and actual hearing which we hardly touched upon in order to move onto expectation bias in wine tasting.  If anyone should be suggesting the direction of the discussion it should be the original poster.

What we measure is what we hear

Reply #46
Maybe a little OT, but I recall a "rule of thumb" more than a woman told me they follow when they buy perfumes is they don't evaluate more than three or even two different fragrances a time, or they would not be able to discern real differences and end up taking home something they could even not like any more days after.
I guess this has everything to do with temporal masking and even perception system fatigue.

P.S. Happy new year full of good music to all HA fellows! 
... I live by long distance.

What we measure is what we hear

Reply #47
Feel free to start a new topic, replacing the word hear in the subject line with the word taste.  Be sure to post it in the off-topic forum.


Actually, I know of at least 1 or 2 good books on sensory evaluation of wine that directly read on audio testing if you think even a little bit.


I'm reminded of Mielgard, et al  classic book about Sensory Evaluation.

http://books.google.com/books?id=F_A-YtWXF3gC

What we measure is what we hear

Reply #48
but if perception really varied from moment to moment, we could as well argue that one day an individual could positively pass an ABX test and the other day fail, expecially at borderline conditions.


There are several different issues. One is that our perception changes from moment to moment. Another is we can't focus on everything all at once when complex music is playing. Another is that some things really are so obvious that they won't be missed whether sighted or blind. Another is expectation affecting our opinion when we know which [device] we're hearing. Another is that moving your head even one inch in any direction really does change the sound reaching your ears (speakers, not headphones). Another is that our sensitivity changes at, as you said, borderline conditions. A real change may be noticed one time but not another. Especially for an old man like me.

--Ethan
I believe in Truth, Justice, and the Scientific Method

What we measure is what we hear

Reply #49
... moving your head even one inch in any direction really does change the sound reaching your ears (speakers, not headphones)
Almost all headphones "suffer" from a similar problem - seating them on/in one's head/ear (or any headphone measurement device). Two people are not very likely to position the same headphone exactly the same way on their heads due to different head/ear shapes, preferences etc. This is going to change the perceived/measured sound and introduce additional variable(s) into the mix.