Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: what value listening tests (Read 41604 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

what value listening tests

Reply #50
omg, I just ran into this. Just substitute "homeopath" for "audiophile". EVERY SINGLE AUDIOPHILE TROPE. (Well, except one, but we're waiting to be accused of government conspirators any time now.)

what value listening tests

Reply #51
Besides, scientists used to think the Earth was flat,  and that Earth, Water, Air and Fire were elements. They don't always get it right, you know.

Asinine - why? What I said is true. Factual. Accurate. Everyone knows this. Scientists are not God. They make many mistakes. Todays great science is tomorrows embarassing nonsense.

false. this is a VERY common misconception. but very wrong. I would even say asinine, except that so many people believe that scientists used to think the earth was flat, that it's easy to accept.

This isn't a big deal, but everyone quotes it all the time to make a point, and as such is an invalid argument.
the classic debate in Galileo's case was whether the sun revolved around the earth or vice versa. but scientists or "natural philosophers" as they were called then and earlier, have never put much stock in the earth's being flat. the pre-eminent medieval astronomy text was entitled "sphere." and understanding of earth as a sphere extends to classical Greek philosophers as well.

Now I do agree that scientists have been wrong on many things. and theories get stuck in paradigms that resist change. etc., etc. these may become embarrassing later on. But much of natural science in the 20th Century hasn't been paradigm shifts and throwing out old paradigms, but rather incremental steps forward, adding nuance, etc. Total reversals and embarrassments don't happen very often.
God kills a kitten every time you encode with CBR 320

what value listening tests

Reply #52
also, it's worth noting that this thread was resolved a long time ago, and the debating since then hasn't gone anywhere. Just the OP appearing to be more of a troll, but not certainly so. (e.g., it may actually be that WMA at 160kbps sounds better to his ears than mp3 at 320kbps. that's not the case for most people, but it's plausible that it could be for some.)

the main question still in play is whether people's brain states differ, for different sorts of listening. if this is true, it will vary between individuals - some people's "critical" filter will probably be always turned on, whether they are ABX testing or not.
But testing could still be set up for this, you'd just need to trick people (which social psych tests to all the time) by testing something different than what people think they're being tested on.

The only thing in this thread that still needs (or deserves) to be addressed is Gag Halfrunt's question:

My original question was regarding neuroimaging - whether 'listening to music' is materially different to 'using music to evaluate something.' Are the two activities neurologically distinct? This was the test that I am not sure if anyone has performed - not 'armchair skeptic', but genuine inquiry.
Does anyone know of a paper on this?
God kills a kitten every time you encode with CBR 320

what value listening tests

Reply #53
also, it's worth noting that this thread was resolved a long time ago, and the debating since then hasn't gone anywhere. Just the OP appearing to be more of a troll, but not certainly so. (e.g., it may actually be that WMA at 160kbps sounds better to his ears than mp3 at 320kbps. that's not the case for most people, but it's plausible that it could be for some.)

the main question still in play is whether people's brain states differ, for different sorts of listening. if this is true, it will vary between individuals - some people's "critical" filter will probably be always turned on, whether they are ABX testing or not.
But testing could still be set up for this, you'd just need to trick people (which social psych tests to all the time) by testing something different than what people think they're being tested on.

The only thing in this thread that still needs (or deserves) to be addressed is Gag Halfrunt's question:

My original question was regarding neuroimaging - whether 'listening to music' is materially different to 'using music to evaluate something.' Are the two activities neurologically distinct? This was the test that I am not sure if anyone has performed - not 'armchair skeptic', but genuine inquiry.
Does anyone know of a paper on this?




It might be that much of this sort of work is done on hearing of speech.  I would assume there must be a physiological/neurological difference at least between nonattentive and attentive listening (e.g. background chatter versus the person who is speaking to you at a party). 

Or perhaps Daniel Levitan's book might have some clues (This Is Your Brain On Music).



what value listening tests

Reply #54
There's a strong possibility that the psychological (or even neurological) conditions that apply when a person is enjoying a piece of art are materially different to those that apply when the same person is studying that artwork for authenticity. The second condition is not necessarily an amplified version of the first.

If that is the case, there might be different perception pathways between a person listening to a news report and the same person listening a piece of poetry, for example. So, does does music 'appreciation' differ from music 'evaluation' and using music as an evaluative tool in audio development?

Here's the problem...

It seems that there's some kind of implication here by some that "appreciation" cannot be a metric to distinguish a difference in a double-blind test.  At the same time these people want to then say there is indeed a difference between sample A vs. sample B for the listener because one makes him feel better than the other.

Sorry, but you can't have it both ways!

what value listening tests

Reply #55
Or perhaps Daniel Levitan's book might have some clues (This Is Your Brain On Music).


Good book. There is an online archive of Levitan's papers at Levitan's papers, some online

The answer to the basic question comparing and contrasting the physiology of listening for pleasure and listening analytically is probably in there somewhere.

He sheds some light on the physiology of listening for pleasure in Levitan's Online Paper About Listening For Pleasure (PDF)

He also seems to be interested in the phenominon of perfect pitch, which might be a good example of listening analytically. One or more of his papers about perfect pitch might be studied to compare and contrast the physiology of listening for pleasure and listening analytically.

what value listening tests

Reply #56
Is anybody going to comment on my post, or did I respond to the wrong question? 

what value listening tests

Reply #57
Or perhaps Daniel Levitan's book might have some clues (This Is Your Brain On Music).


Good book. There is an online archive of Levitan's papers at Levitan's papers, some online

The answer to the basic question comparing and contrasting the physiology of listening for pleasure and listening analytically is probably in there somewhere.

He sheds some light on the physiology of listening for pleasure in Levitan's Online Paper About Listening For Pleasure (PDF)

He also seems to be interested in the phenominon of perfect pitch, which might be a good example of listening analytically. One or more of his papers about perfect pitch might be studied to compare and contrast the physiology of listening for pleasure and listening analytically.


Thanks for this... Levitan's book came out about the same time my father passed and it kind of got pushed aside. I read Oliver Sacks book, but it was not very informative, IMO. I'm a bit wary of both because I dislike pop-science.

His papers seem usefully academic. Will check out.

what value listening tests

Reply #58
Here's the problem...

It seems that there's some kind of implication here by some that "appreciation" cannot be a metric to distinguish a difference in a double-blind test.  At the same time these people want to then say there is indeed a difference between sample A vs. sample B for the listener because one makes him feel better than the other.

Sorry, but you can't have it both ways!


No implication implied.

My guess (and it is just a guess at this time) would be the 'evaluation' part only kicks in when someone is listening to evaluate. At all other times, the 'appreciation' pathway holds. That would mean neither should interact with each other, and is likely why someone can appreciate a piece of music qua music even when the signal quality is heavily degraded.

The interesting part (for me at least) is whether these two potentially different neurological pathways ever clash. More significantly, do they have a 'use it or lose it' quality... audiophiles often seem so obsessed with quality of sound that they seem to forget there's a musical component to audio. I'm just wondering if this is something that eventually hardwires, making it harder for someone who spends their life evaluating sound to appreciate music under any circumstances.

I know a lot of audiophile nonsense is basically OCD with a credit rating, but those twitchy, unable to sit through a piece of music without changing something audiophiles appear to demonstrate a fundamental inability to separate music appreciation from audio evaluation.

I'm conscious that this is thread crapping, but OTOH, it's a crap thread!

what value listening tests

Reply #59

also, it's worth noting that this thread was resolved a long time ago, and the debating since then hasn't gone anywhere. Just the OP appearing to be more of a troll, but not certainly so. (e.g., it may actually be that WMA at 160kbps sounds better to his ears than mp3 at 320kbps. that's not the case for most people, but it's plausible that it could be for some.)


Would somneone please explain what it is about my post that makes me a troll? I am not, but obviously there is something I am doing that makes people think i am - perhaps you could help?

Its not so much about how much distortion a codec has intorduced in total, but about the different types of distortion and which ones you are more sensitive to. If I had to do an a/b between 160kbs wma and the cd source versus 320kbs mp3 and the cd, I do not doubt it would be much easier to notice the difference with the 160kbs wma than the 320kbs mp3. However, for me, the mp3 is never enjoyeable to listen to - there must be some aspect of the sound my subconcious is very sensitive to.

I am very sensitive to anything that diminishes the transient attack and clarity of music, more so than almost anything else. I can put up with large tonal variations for example - too little bass, prominent midrange, etc, things that are real easy to spot in an a/b comparision -  but provided the details in the music sound clear and the soundstage is not blurred, I can adjust to the other deficiencies. But if the soundstage detail and transient attack is blurred, I cannot lenjoy istening to the music, no matter how good it is in all other respects.

Thats one reason I am making this post, to note that there are some differences that are easy to pick up in listening tests, and some that are much harder. However, often the more subtle and difficult to spot differences have more impact on the music than the more obvious things. IF for example a codec adds even a small amount of boost to the treble, it can be very easy to spot - at least it is for me. But then this would not bother me - the frequency response of the system you are listening on is never flat, and their are treble tone controls. But slight changes to transient attack may be, for those not like myself who are sensitised to it, much harder to spot. They are also much harder to deal with - there are no "transient attack" controls on the mp3 players.

I will plainly admit that I find it hard to tell the difference between 320kbps mp3 and a cd source, especially with certain kinds of material, but I do know that the pleasure centre of my brain seems to be able to do it subconciously, since I never sem to enjoy listening (as opposed to scrutinising) to mp3's.

what value listening tests

Reply #60
Besides, scientists used to think the Earth was flat,  and that Earth, Water, Air and Fire were elements. They don't always get it right, you know.

Asinine - why? What I said is true. Factual. Accurate. Everyone knows this. Scientists are not God. They make many mistakes. Todays great science is tomorrows embarassing nonsense.

false. this is a VERY common misconception. but very wrong. I would even say asinine, except that so many people believe that scientists used to think the earth was flat, that it's easy to accept.

This isn't a big deal, but everyone quotes it all the time to make a point, and as such is an invalid argument.
the classic debate in Galileo's case was whether the sun revolved around the earth or vice versa. but scientists or "natural philosophers" as they were called then and earlier, have never put much stock in the earth's being flat. the pre-eminent medieval astronomy text was entitled "sphere." and understanding of earth as a sphere extends to classical Greek philosophers as well.

Now I do agree that scientists have been wrong on many things. and theories get stuck in paradigms that resist change. etc., etc. these may become embarrassing later on. But much of natural science in the 20th Century hasn't been paradigm shifts and throwing out old paradigms, but rather incremental steps forward, adding nuance, etc. Total reversals and embarrassments don't happen very often.


In future when I make this kind of quote, I will say "Scientists used to think the sun revolved around the Earth", or "Scientists used to think heat was a weightless fluid called caloric". Actually I chose this example because I thought more people would be familiar with it, and also I did'nt realise how pedantic people would be, and the modern tendency to for people to nit pick everything to bits.

?

what value listening tests

Reply #61

So long as it keeps psychiatrists and psychologists in business then I'm all for it.

Seriously though, I wasn't trying to single you out and I do think you have a very good point as does the common sense notion that the "evaluation" part involves a higher level of scrutiny and greater attention to detail as opposed to the "appreciation" part.  Nonsensically, the OP is actually trying to suggest the opposite is the case.

To be candid, my comment was actually aimed at the recent developments in the other thread that is currently active on the same topic.  A recent poster seems to be trying to attach credibility and justification for some of the night and day subjective nonsense that we commonly see: "Additionally, long term exposures and subsequent judgments might reveal preference differences between stimuli that are also not perceptually different using ABX. If this is the case (and I suspect it very well could be), then short term and/or long-term processing differences might be at the root of some people's apparent discomfort associated with lossy codecs, and even 44.1/16 bit digitization."

what value listening tests

Reply #62
I will plainly admit that I find it hard to tell the difference between 320kbps mp3 and a cd source, especially with certain kinds of material, but I do know that the pleasure centre of my brain seems to be able to do it subconciously, since I never sem to enjoy listening (as opposed to scrutinising) to mp3's.

This is the placebo effect, it has nothing to do with your subconscious.

If I had to do an a/b between 160kbs wma and the cd source versus 320kbs mp3 and the cd, I do not doubt it would be much easier to notice the difference with the 160kbs wma than the 320kbs mp3.

As an act of good faith, let's see some ABX logs, magneticrabbit.

what value listening tests

Reply #63
When I put wma versions of the same music on my player, I was suddenly interested once again, and enjoying the music once again. I actually performed this test by accident, accidentally replacing the wma with mp3, which i discovered later -  a perfect example of unbiased testing.

Is this an admission that since this was just an accident and that all the rest of your so-called "a/b" testing was sighted?

I'll take your lack of response to this question as a yes.  All the rest of your posts have given me no reason to believe otherwise.

what value listening tests

Reply #64

So long as it keeps psychiatrists and psychologists in business then I'm all for it.

Seriously though, I wasn't trying to single you out and I do think you have a very good point as does the common sense notion that the "evaluation" part involves a higher level of scrutiny and greater attention to detail as opposed to the "appreciation" part.  Nonsensically, the OP is actually trying to suggest the opposite is the case, though it is clear that this individual is clueless about scientific method.

To be candid, my comment was actually aimed at the recent developments in the other thread that is currently active on the same topic.  A recent poster seems to be trying to attach credibility and justification for some of the night and day subjective nonsense that we commonly see: "Additionally, long term exposures and subsequent judgments might reveal preference differences between stimuli that are also not perceptually different using ABX. If this is the case (and I suspect it very well could be), then short term and/or long-term processing differences might be at the root of some people's apparent discomfort associated with lossy codecs, and even 44.1/16 bit digitization."


I guess there is a consderable incentive to Hydrogen Audio forum supporters to reject the notion that sound quality is too ephemeral and subjective to ever properly quantify, and the brain too poor an analytic tool to accurately measure these things we cannot see, since if it were true some part of the raison d'etre of Hydrogen Audio would vanish in a puff of smoke.- or at least thats how it seems to me at the moment. Hydrogen Audio sems to me to be very "pro-test".

And I guess no way you can every change anyones opinion on this - how can you prove that something cannot be proved? People that believe whole heartedly in the scientific method will not accept any proof that the scientific method does not work unless the proof is in the form of the scientific method. Catch-22 I think.

what value listening tests

Reply #65
Yawn! (reprise)

As has been said before, I claim there is a pink elephant in orbit around the planet Uranus.  Until you can prove I am wrong, my claim must be true.

PS: I see you caught my post before my edit.  While it is painfully obvious that you are clueless about scientific method and even seem to reject it in favor of false world view based on blind and unprovable faith, it really had nothing to do with my point which is why I removed it.

what value listening tests

Reply #66
When I put wma versions of the same music on my player, I was suddenly interested once again, and enjoying the music once again. I actually performed this test by accident, accidentally replacing the wma with mp3, which i discovered later -  a perfect example of unbiased testing.

Is this an admission that since this was just an accident and that all the rest of your so-called "a/b" testing was sighted?

I'll take your lack of response to this question as a yes.  All the rest of your posts have given me no reason to believe otherwise.


My test was the best kind of test ever. Not only was it "unsighted", but I was not even aware I was conducting a test. It was a non-test test. How can you improve on that?

Since I was not conciously conducting a test, but simply doing normal listening, it was an absolutely pure situation, unaffected by any unnatural external infuences. Far superior to any kind of normal test where the listeners know they are in a test - no matter how scientifically you conduct it.

This is why I place so much stock in what happened.

I would have thought you would have loved this situation - its the ideal for all listening tests - a test where the listener did'nt even know it was a test.

As someone who has been listening to high end hi-fi since I was in my early teens, I long ago abandoned listening tests for selecting hi-fi systems. I found it impossible to relax in hi-fi shops when auditioning equipment, and unable to make any meaningful judgements about sound quality under those forced conditions. Since I place no real value in listening tests, all my other tests were sighted, yes. Why not? I was not performing the tests for anyone other than myself. Nearly all my other tests were not however codec comparisons, but bit rate comparisons, the object being to determine if raising the bitrate led to any appreciable gains in quality. This is was in the days when my mp3 player only had a few gigs of storage on it.

But most of my tests were not forced A/B tests - you might have noticed i don't believe in them - but rather I would convert all my music to a particular codec, and then listen normally for some days, and see if I was enjoying the music, and if I was noticing any deficiencies in the sound. Try one codec for a couple of days, and then switch to another. Was that better, or worse? Thats what I do, and it seems to work well - for me.

On this basis, I selected the wma codec. I find that when I have my music encoded in that format, I enjoy listening to the music. Week after week, month after month. I find if I encode the music in mp3, i almost immediately lose interest in listening - over days, weeks or months. Its not a mood swing on any particular day. I have no reason to dislike mp3 per se - in fact every reason to like it - it has the most support. And as noted above, I disliked listening to mp3 even when I thought i had wma on the player but had mp3 by accident, suggesting that its not any form of auto-suggestion.

And yet many people on this forum are telling me I should do an a/b unsighted test that will reveal that I cannot tell the difference between the mp3 and the cd source, and force myself to listen to mp3, despite hating the sound, because its scientifically proven that its as good as the source. It would'nt work. No matter how much scientific proof you produced that the 320kbs mp3 was so near the cd that it would be hard to tell them apart (for me to), I still would hate the sound. Thats all there i to it. QED.


I would also like to add that I hate Microsoft with a passion, and so I was very peed off when I found that I preferred the wma codec. In addition, I love the ideals of open source and use Linux in addition to Windows when I can (more of that Microsoft hatred again i suppose), and so have a natural inclination to try and use Vorbis or other free codecs when I can, like MPC. So the fact that I end up selecting wma is not due to some subconcious inclination, but based purely on what sounds best to me.

what value listening tests

Reply #67
if it were true some part of the raison d'etre of Hydrogen Audio would vanish in a puff of smoke.

Feel free to try, though I won't be holding my breath.

what value listening tests

Reply #68
My test was the best kind of test ever. Not only was it "unsighted", but I was not even aware I was conducting a test. It was a non-test test. How can you improve on that?
By repeating it.  Otherwise it is completely dismissible.

Far superior to any kind of normal test where the listeners know they are in a test - no matter how scientifically you conduct it.
Your opinion is noted and rejected.

This is why I place so much stock in what happened.
Go back and read my anecdotal comment about being able to predetermine the result of a coin-flip.

what value listening tests

Reply #69
My test was the best kind of test ever. Not only was it "unsighted", but I was not even aware I was conducting a test. It was a non-test test. How can you improve on that?

Since I was not conciously conducting a test, but simply doing normal listening, it was an absolutely pure situation, unaffected by any unnatural external infuences. Far superior to any kind of normal test where the listeners know they are in a test - no matter how scientifically you conduct it.

This is why I place so much stock in what happened.


The problem here is that it's just one test. Did you get the 'right' result because your test demonstrates something that does not parse under other conditions, or did you get the result through pure chance?

Accidentally repeat this test enough times to be statistically significant and you have a chip in the game. Otherwise, you're still playing for matches.

what value listening tests

Reply #70
To be candid, my comment was actually aimed at the recent developments in the other thread that is currently active on the same topic.  A recent poster seems to be trying to attach credibility and justification for some of the night and day subjective nonsense that we commonly see: "Additionally, long term exposures and subsequent judgments might reveal preference differences between stimuli that are also not perceptually different using ABX. If this is the case (and I suspect it very well could be), then short term and/or long-term processing differences might be at the root of some people's apparent discomfort associated with lossy codecs, and even 44.1/16 bit digitization."


I think this is a new form of audiophile dogma. As ABX testing doesn't give the result they are looking for and AB testing is just lobotomized ABX, they've regrouped and come up with a new 'test'. This time with extra magic.

I would imagine long-term exposure to a thing would only serve to help you 'learn to love' that thing. Isn't that what 'puppy dog' selling is all about?

what value listening tests

Reply #71
Yawn! (reprise)

As has been said before, I claim there is a pink elephant in orbit around the planet Uranus.  Until you can prove I am wrong, my claim must be true.

PS: I see you caught my post before my edit.  While it is painfully obvious that you are clueless about scientific method and even seem to reject it in favor of false world view based on blind and unprovable faith, it really had nothing to do with my point which is why I removed it.


I never said it must be true. It is a hypothesis - a theory. Surely I can present a theory for others to chew over, simply for the fun of it, and to put new ideas into the mix. As I have said before, I thought i was submitting an idea to a chat forum, not publishing a scientific paper in Nature magazine. This is a simple chat, not Cold Fusion.

 

what value listening tests

Reply #72
My test was the best kind of test ever. Not only was it "unsighted", but I was not even aware I was conducting a test. It was a non-test test. How can you improve on that?

Since I was not conciously conducting a test, but simply doing normal listening, it was an absolutely pure situation, unaffected by any unnatural external infuences. Far superior to any kind of normal test where the listeners know they are in a test - no matter how scientifically you conduct it.

This is why I place so much stock in what happened.


The problem here is that it's just one test. Did you get the 'right' result because your test demonstrates something that does not parse under other conditions, or did you get the result through pure chance?

Accidentally repeat this test enough times to be statistically significant and you have a chip in the game. Otherwise, you're still playing for matches.


Agreed. But then since it was an accident and not a planned test, how could it be repeated?

I suppose I could get my son or daughter to preload my mp3 player with a different sd card each morning without telling me which one - but then I would still know I was in a test, and my subsequent listening would be conditioned by that knowledge. hence, the test would be useless to me, since as I have stated in my post, that would, for me, invalidate the test. My basic principle for listening tests is that once you know you are being tested - the test is invalid.

what value listening tests

Reply #73
My test was the best kind of test ever. Not only was it "unsighted", but I was not even aware I was conducting a test. It was a non-test test. How can you improve on that?
By repeating it.  Otherwise it is completely dismissible.

That is a very revealing comment. That the tests you are conducting are not reliable unless repeated. That suggests that the test is in itself is very unreliable, hence the need to repeat it to avoid false results. If the differences being detected were easy to detect, repetition would not be necessary i think.

Far superior to any kind of normal test where the listeners know they are in a test - no matter how scientifically you conduct it.
Your opinion is noted and rejected.

This is why I place so much stock in what happened.
Go back and read my anecdotal comment about being able to predetermine the result of a coin-flip.


Quite, but then hwo repeat a test that was an accident and not a planned test? You see my dilemma.

what value listening tests

Reply #74
People that believe whole heartedly in the scientific method will not accept any proof that the scientific method does not work unless the proof is in the form of the scientific method. Catch-22 I think.


I have a real problem with this oft-repeated nonsense. This would be bad enough if you were advocating astrology or Reiki or some other New Age idiocy. However, audio components are (or at least were) built using applied science. Audiophile brands sometimes have a loose understanding of that applied science, often preferring instead to take a well-engineered circuit from a legitimate electronics engineer and pimp it out with huge transformers and capacitors. But even the most fluffy of audiophile amplifier designer will know his or her way around Ohm's law.

Last time I looked, Sony had not appointed a design shamen to help develop its next generation of OLED televisions, Nikon does not employ someone to channel positive energy into its D3S camera body to allow it to work at high ISOs and Panasonic's Blu-ray players do not have to be inspected by white witches before final sign off. What makes the reproduction of sound so special that it tweaks the nose of science and laughs in the face of engineering when all these other consumer electronic disciplines don't?