Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: what value listening tests (Read 42018 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

what value listening tests

Reply #25 –
Quote from: timcupery on
To both of these I say: awesome. thanks for the well-framed comments, greynol and Light-Fire

I'm not so sure about the framing, but I think my comment is pretty spot-on; and not only having the OP conduct such a test himself, there's a good chance he won't even come back with a follow-up post.

Some people seem to want to dismiss placebo effect as a possible reason for subjective differences in favor of some other phenomena that doesn't hasn't been reliably demonstrated to exist to the extent that placebo effect has.

I don't need to remind everyone of JJ's presentation on steering in Ethan Winer's Audio Myths Workshop, do I?

what value listening tests

Reply #26 –
Quote from: krabapple on
Blind tests where the subjective *preference* or emotional response rather than difference is evaluated, are such tests: MUSHRA, etc. 


This is my problem with armchair 'skeptics' in many fields.  They seem to think no scientist in the field could have considered their point before.


(OTOH, if you're going to complain that no one has performed EXACTLY a certain test you might have in mind -- e.g., in THIS PARTICULAR living room with THIS PARTICULAR piece of gear/music -- please don't bother.)


Hmm, I think I diluted my own question.

My original question was regarding neuroimaging - whether 'listening to music' is materially different to 'using music to evaluate something.' Are the two activities neurologically distinct? This was the test that I am not sure if anyone has performed - not 'armchair skeptic', but genuine inquiry.

Does anyone know of a paper on this?




what value listening tests

Reply #27 –
Quote from: Gag Halfrunt on
...My original question was regarding neuroimaging - whether 'listening to music' is materially different to 'using music to evaluate something.' Are the two activities neurologically distinct? This was the test that I am not sure if anyone has performed - not 'armchair skeptic', but genuine inquiry.

Does anyone know of a paper on this?


ABX listening tests performed in this forum have the objective of finding out how much can we reduce the file sizes without incurring in audible differences. So we can use that information to efficiently encode our music and listen to it later without any audible artifact.

Contrary to what you may think. It is way easier to find a problem when performing and ABX than it is when casually listening to music.

what value listening tests

Reply #28 –
Quote from: Light-Fire on
Contrary to what you may think. It is way easier to find a problem when performing and ABX than it is when casually listening to music.

I'm in full agreement with this (or am I actually saying the opposite?) purely on the basis that I know I can ABX better in the comfort of my own home on familiar audio equipment than I would at an official meeting of some kind.

If any of my other senses are stimulated to any noticeable degree then I'd be suspicious that any of those could have created a temporary mask via distraction to my auditory senses. I've found that the most accurate ABX-ing I've carried out have been those of minimum distraction. I'm not busting a blood vessel by forcing myself to listening so hard that my ears bleed, I'm just listening.

Maybe I'm atypical and it doesn't work for everybody else in the same way that it works for me, but give me a comfortably warm, relaxingly darkened room on a familiar sofa shortly after an evening meal for my ABX tests anyday. The less distractions from my auditory pleasures the better, thank you very much.

what value listening tests

Reply #29 –
Quote from: pdq on
Let us not overlook that the arguments presented by the OP are typical of someone who believes he hears a difference in a sighted test, but is unable to prove it in a proper double-blind test.

The bottom line is that if you design the test in a way that you think will prove that you hear a difference, and as long as it is a proper double-blind test and the results are statistically significant, then we will accept that you can indeed hear a difference. Anything less will not be accepted.


Proof in a test means nothing if the test is not valid. Of course, if someone like myself attacks the validity of the test, one can easily make the charge that he is only doing it because he has failed the test. Its a nice circular kind of reasoning, although wrong in my case. If you cannot tell the difference between two test sounds in an a/b codec comparison, does it mean the codecs are equally good, or does it indicate a lack of discernment in the listener? Since this is all subjective, no one can ever really know.

I am amazed that with something as nebulous and hard to define as the sensation of music (i defy anyone to describe what it is or disseminate it in any meaningful way), people are so sure you can stick it under a microscope and examine it as if it were something visible. All I can say is that these a/b tests are not much of a microscope. The sound of a hi-fi systems changes even with the humidity content in the air, the air temperature, dust particles on the tweeter cone, etc, but if you want to believe you can pin down the subtleties of sound in this way, then jolly good luck to you.

what value listening tests

Reply #30 –
Quote from: andy o on
I don't know if this topic is gonna be deleted, cause the "argument" has been said a million times before.

[


Its nice to know that millions of people agree with me. I thought I might be out on my own with this one.

 

what value listening tests

Reply #31 –
Quote from: magneticrabbit on
Quote from: andy o on
I don't know if this topic is gonna be deleted, cause the "argument" has been said a million times before.

[


Its nice to know that millions of people agree with me. I thought I might be out on my own with this one.





So, you think it matters that 'millions' of people agree, without specifying what those people actually *know*?  You sure you want to go there? 


Go away , troll. 

what value listening tests

Reply #32 –
Quote from: richard123 on
A pill without any active ingredient can nonetheless cause a physiological change in a person.  Bitrate information can have an effect on listeners, even without a basis in reality.  Discussions of ABX testing seem to dismiss this effect too quickly.


No, they don't.  They only 'dismiss'  (or more carefully stated, support or fail to support)  that the ACTUAL (versus BELIEVED) state of A or B is what makes the difference. 

I appears you don't understand what DBTs are.  That's OK, but it's not OK to make claims about them, in that case.

what value listening tests

Reply #33 –
Quote from: pdq on
Nobody here cares how you evaluate what you are listening to as long as you meet one critical requirement. You must not know which version you are listening to. Beyond that there are many many valid approaches to "listening tests".


As you rightly point out, not knowing in advance which version you are listening to is one important aspect of listening tests, in effect taking steps to avoid prejudicing the listener, and shows that people understand the importance of the listeners state of mind during the test and how easy it is for them to be influenced by external factors.

Unfortunately, as I have said in my post, the one factor you cannot filter out and which can have a very large influence on the mood of the listener, is the fact that they are in a listening test. Until you can devise a listening test where those taking part are unaware of the fact they are in a listening test (impossible perhaps?), no test will ever be unbiased or natural.

For a test to be properly scientific, ALL unusual external factors need to be accounted for, not just some.

what value listening tests

Reply #34 –
Quote from: krabapple on
Quote from: magneticrabbit on
Quote from: andy o on
I don't know if this topic is gonna be deleted, cause the "argument" has been said a million times before.

[


Its nice to know that millions of people agree with me. I thought I might be out on my own with this one.





So, you think it matters that 'millions' of people agree, without specifying what those people actually *know*?  You sure you want to go there? 


Go away , troll.


I raised an argument - you said millions of others have raised the same argument - that suggest that millions of others agree with me. That simple.

what value listening tests

Reply #36 –
Quote from: Light-Fire on
Quote from: magneticrabbit on
Its nice to know that millions of people agree with me. I thought I might be out on my own with this one.


Science does not agree with you.
 


I am not really interested in who does or not agree with me. Its the subject I am interested in, not people. What matters to me is the validity of my argument, or lack thereof. And for someone to prove to me its false, I have to see some eveidence in the form of reasoned argument. IF I am shown its false, I will retract it.

Besides, scientists used to think the Earth was flat,  and that Earth, Water, Air and Fire were elements. They don't always get it right, you know.

what value listening tests

Reply #37 –
Quote from: magneticrabbit on
Besides, scientists used to think the Earth was flat,  and that Earth, Water, Air and Fire were elements. They don't always get it right, you know.

no, those were philosophers and you are lacking basic school knowledge for 10y kid

what value listening tests

Reply #38 –
Quote from: magneticrabbit on
I am not really interested in who does or not agree with me. Its the subject I am interested in, not people. What matters to me is the validity of my argument, or lack thereof. And for someone to prove to me its false, I have to see some eveidence in the form of reasoned argument. IF I am shown its false, I will retract it.

Why the hell did you bring up that millions of people agree with you then, as if that had any validity? You're "not interested in who does or does not agree" with you, but you are interested in how many do agree with you? Reality is not decided by votes.

what value listening tests

Reply #39 –
Quote from: magneticrabbit on
...Its the subject I am interested in, not people. What matters to me is the validity of my argument, or lack thereof. And for someone to prove to me its false, I have to see some eveidence in the form of reasoned argument. IF I am shown its false, I will retract it.



You are the one that have to prove your argument is right.

Why don't you try some codec ABXing to find out how "golden" are your ears? It is an "eye-opening" experience.

what value listening tests

Reply #40 –
Quote from: magneticrabbit on
I am not really interested in who does or not agree with me. Its the subject I am interested in, not people. What matters to me is the validity of my argument, or lack thereof. And for someone to prove to me its false, I have to see some eveidence in the form of reasoned argument. IF I am shown its false, I will retract it.

The custom is normally that if you make an argument, you also provide the proof for it. Since you haven't provided any so far, I feel free to dismiss it.

If I read it correctly your argument can be summarized that you have a bigger chance of finding a difference when you are paying less attention to what you are listening. This is at least seems to be contradiction, so it's up to you to provide the proof.

And no, I don't take examples from your personal life as proof.
"We cannot win against obsession. They care, we don't. They win."
 

what value listening tests

Reply #41 –
Quote
For a test to be properly scientific, ALL unusual external factors need to be accounted for, not just some.

To a limit of practicality. How can you ask someone to evaluate whether or not they hear a difference, without asking them whether or not they hear a difference?

Quote
Besides, scientists used to think the Earth was flat, and that Earth, Water, Air and Fire were elements. They don't always get it right, you know.

Utterly asinine.

what value listening tests

Reply #42 –
Quote from: magneticrabbit on
Besides, scientists used to think the Earth was flat,  and that Earth, Water, Air and Fire were elements. They don't always get it right, you know.


Huh, Metaphysics is not Science. It is a branch of philosophy.

what value listening tests

Reply #43 –
Quote from: Light-Fire on
Quote from: magneticrabbit on
...Its the subject I am interested in, not people. What matters to me is the validity of my argument, or lack thereof. And for someone to prove to me its false, I have to see some eveidence in the form of reasoned argument. IF I am shown its false, I will retract it.



You are the one that have to prove your argument is right.

Why don't you try some codec ABXing to find out how "golden" are your ears? It is an "eye-opening" experience.


I have ABX'd codecs on many occasions. Thats why I use almost exclusively WMA - it seems to handle transients the best. I nearly always max out the bitrate, since sound quality is high on my list of priorities. At least I never go below about 160kbits even when I need to cram lots of stuff in a small space. Fortunately all my players bar one have SD slots, and with SD cards now hitting 32gigs, there is less and less need to do high compression. I find MP3 to be the worst codec of the lot. I would rather have WMA at 160kbits than MP3 at 320k. MP2 seems way better than MP3 to me. My current codec preference is roughly WMA, MP2, MPC, AAC / Vorbis, ------------------------------------------> MP3. I have never heard a codec at any bitrate that sounds as good as a CD. They sound close with a cursory listen, but extended listening on good equipment always reveals the full truth. But a good codec at a high bitrate can get close enough to work well on a mid-fi DAP.

One reality that golden ears shows you with music is that the most professional recordings have only so-so sound quality. The small number of excellent recordings are greatly outnumbered by the average ones. It does not matter what sort of codec you use - a bad recording is a bad recording - aint nothing you can do about that. And there are so many bad ones.

I don't understand why you say I have to prove myself right. I am not trying to prove anything. I have formed a theory which I believe is correct (otherwise I would'nt be writing about it), and am communicating this idea to others for their consideration. They are at liberty to read it, ignore it, laugh at it, or even print it out and wipe their bum with it. I could care less one way or the other.

If someone is convinced I am wrong, and wants to try and convince me of their position, I will be glad to hear about it, and as I say, if I feel they have proved me wrong, I will retract my former opinion and aknowledge the correctness of their argument. If I do not find their argument convincing, I will say why. The cycle may then repeat, depending upon the other persons reaction to my rebuttal.

But as to proving myself right - I really have no interest in doing that. Which is probably fortunate, since in my past experience, no one ever admits they are wrong, so I would have little chance of doing so anyhow.

what value listening tests

Reply #44 –
Quote from: dv1989 on
Quote
For a test to be properly scientific, ALL unusual external factors need to be accounted for, not just some.

To a limit of practicality. How can you ask someone to evaluate whether or not they hear a difference, without asking them whether or not they hear a difference?

Quote
Besides, scientists used to think the Earth was flat, and that Earth, Water, Air and Fire were elements. They don't always get it right, you know.

Utterly asinine.


Agreed you can only do whats practical. Nonetheless, this inability to fully isolate the listener from the environment of the test still has an impact, whether you like it or not. Just because you cannot solve a problem, does not mean the problem will conveniently go away.

If I had thought there was a way to solve this problem, I would have suggested it. I can't, just as you can't, which is why I say that listening tests are not really as useful as you might think.

Asinine - why? What I said is true. Factual. Accurate. Everyone knows this. Scientists are not God. They make many mistakes. Todays great science is tomorrows embarassing nonsense.



what value listening tests

Reply #45 –
Quote from: stephanV on
Quote from: magneticrabbit on
I am not really interested in who does or not agree with me. Its the subject I am interested in, not people. What matters to me is the validity of my argument, or lack thereof. And for someone to prove to me its false, I have to see some eveidence in the form of reasoned argument. IF I am shown its false, I will retract it.

The custom is normally that if you make an argument, you also provide the proof for it. Since you haven't provided any so far, I feel free to dismiss it.

If I read it correctly your argument can be summarized that you have a bigger chance of finding a difference when you are paying less attention to what you are listening. This is at least seems to be contradiction, so it's up to you to provide the proof.

And no, I don't take examples from your personal life as proof.


This is a chat forum, not a court of law. If I wanted to prove this idea to the world, I would have published it in a science journal. I am not interested very much if you believe it or not. What you do with the info is up to you. I am presenting it in a chat forum simply to have a .... chat? By all means feel free to ignore it.

My personal life example was to show you why I believe what i do, not too convince you. As to proof - how can you ever prove what I am saying? Something that is totally subjective? If you could figure out how to do that, I would love to hear about it.

If you have a fully double blind listening test which has the result that AAC is a better codec than say, WMA - is that proof? The subjective judgements of a handful of ordinary human beings like myself? That proves nothing, except perhaps tha the people in the test are no judge of good sound. How can you tell who is a good judge of sound - do they have a sign on their forehead or something?

Believe me, the word "proof" does not belong in any conversation about sound quality. Its just far too subjective.

what value listening tests

Reply #46 –
Quote from: magneticrabbit on
Asinine - why? What I said is true. Factual. Accurate. Everyone knows this. Scientists are not God.
Who said they were?
Quote
They make many mistakes.
Yes, like all other Homo sapiens. Difference is that science thrives from mistakes, as it learns from them. Even more so, trying to falsify others' (and ONE'S OWN) hypotheses is a key element. Pseudoscience peddlers such as relativists and New Agers and "postmodernists" are exactly the opposite. They cling to their mistakes.

Quote
Todays great science is tomorrows embarassing nonsense.
As you've already been told, what you said was not science. Science also progresses, and as time passes it gets more and more solid. Why don't you talk about Newton instead? Or Darwin? They did probably the most important science, centuries ago. Is it embarrassing nonsense yet? If you think the science of today is going to be embarrassing nonsense tomorrow, what the hell are you doing using a computer, and presumably cellphones, cars, airplanes... You don't want to be one of the red-faced idiots when it's proven that the principles that make them work are embarrassing nonsense, do you?

Pseudoscience babblers and relativists, "science has been wrong before, therefore it can't be right" people, may not spout embarrassing nonsense, just because they have proven to have an uncanny inability to be embarrassed.

what value listening tests

Reply #47 –
Quote from: Light-Fire on
Quote from: Gag Halfrunt on
...My original question was regarding neuroimaging - whether 'listening to music' is materially different to 'using music to evaluate something.' Are the two activities neurologically distinct? This was the test that I am not sure if anyone has performed - not 'armchair skeptic', but genuine inquiry.

Does anyone know of a paper on this?


ABX listening tests performed in this forum have the objective of finding out how much can we reduce the file sizes without incurring in audible differences. So we can use that information to efficiently encode our music and listen to it later without any audible artifact.

Contrary to what you may think. It is way easier to find a problem when performing and ABX than it is when casually listening to music.


With the greatest of respect, you are missing the point. I'm interested in the neuroscience behind different kinds of listening and whether there is any neurological change in the perceptive states of listening. I want to know if any research has been done on this. I've searched and cannot find anything, except some recent papers about fMRIs performed on musicians while performing and research on brain injury and musical perception.

My reasons for this interest are entirely unrelated to ABX testing - one of the major symptoms my father noted as he was dying of his glioblastoma was a complete change in the way he perceived music. He lost his emotional connection to music entirely in the space of an afternoon, saying that he could only perceive it as a technical exercise. He was a professional pianist and composer, and his new-found musical 'autism' (his words) did not affect his playing at the time (that went later), but it affected his appreciation of his playing. If anything, his playing became wildly emotional, even though he couldn't perceive it as such. This radically changed the music he would listen to (goodbye Bartók, hello Bach) for the remainder of his life. Reflecting on his death, the change caused by having something the size of a baseball growing in his head seemed to have a parallel in the way music lovers listen to music and the way audiophiles listen to the sound of music.

I find it strange that this seems a non-topic.

what value listening tests

Reply #48 –
Quote from: magneticrabbit on
My personal life example was to show you why I believe what i do, not too convince you. As to proof - how can you ever prove what I am saying? Something that is totally subjective? If you could figure out how to do that, I would love to hear about it.

We can start with proving that you can hear a difference between codec X and codec Y. This is an objective matter.

Quote
If you have a fully double blind listening test which has the result that AAC is a better codec than say, WMA - is that proof? The subjective judgements of a handful of ordinary human beings like myself? That proves nothing, except perhaps tha the people in the test are no judge of good sound. How can you tell who is a good judge of sound - do they have a sign on their forehead or something?

Statistic always have an margin of error. If we make the assumption that the test is intended for the general populous, than a large enough random sample of the general populous will be a good judge of sound.

Quote
Believe me, the word "proof" does not belong in any conversation about sound quality. Its just far too subjective.

You are confused here I'm afraid. I'm not asking proofs about sound quality. I'm asking for proof for your original claim that one can hear differences better under non-listening test conditions, whatever you make those up to be. If you do not wish to discuss this statement then why bring it up in the first place?
"We cannot win against obsession. They care, we don't. They win."

what value listening tests

Reply #49 –
Quote from: magneticrabbit on
This is a chat forum, not a court of law.

I'm sorry that you are under the false impression that this is a chat forum. Perhaps you had better go back and read the terms of service, especially the part about making unsubstantiated claims.