Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: How do you listen to an ABX test? (Read 344437 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

How do you listen to an ABX test?

Reply #150 –
Quote from: greynol on
just like mzil passing your bullet-proof test?  meh.


Mzil was rattling my cage. ;-)

According to his PM, the positive results were obtained while listening to the test tones at the end (t > 12.5 seconds), not during  the keys jangling experiment.

Thst would seem to disqualify his monitoring system.  However, he says that atypically high listening levels were required to reliable detection of audible artifacts in his monitoring system, so all might be well with his monitoring system for the purpose at hand.

His true results for the actual keys jangling listening test segment would then be random guessing. He said that a few posts back. 

The listening test uses linear phase, minimal-ringing, minimal aliasing downsampling, and perceptually shaped dither.

So my bullet-proof test might appear to still be bullet proof (up to this  minute!), with the added features of protecting itself from substandard monitoring systems and unrealistically high playback levels.

Of course we can't do anything about people who don't play the game, which is a truism that grants the power to dismiss what we will based on our perceptions of the person reporting the test results.

Its all good, so far!

How do you listen to an ABX test?

Reply #151 –
Quote from: jkeny on
I'm giving you what I think the widespread opinion of ABX is, as I've experienced it on audio forums -

a) it relies on echoic memory


Can be true, or not.

Quote
b) it deprives the listener of visual knowledge of which device/track is being used.


Right and that is where most claims of reducing bias hang their hat.

Quote
According to the typical proponent of DBTs both of these pillars ensure that DBTs are the "gold standard" of audio tests.


Doesn't make sense on point (1).  Why would relying on  echoic memory be a good thing? It's an obvious debating trade ploy - make your opponent's position as narrow as possible for easier demolition.

Contrary to the Golden Ear's closely held religious beliefs, the ABX developers did not invent quick switching. It was alive and well long before that.  The actual genesis of the original ABX box was that it was a controller for a sighted  quick-switch relay box that had existed for years before.

Quote
I'm not saying that this is correct - I'm just reporting what I have gathered over many years of being on audio forums.


If the futility of looking for truth on your typical audio forum needs underscoring, go over to AVS and have a good laugh at a guy who is trying to convince the world that the percentage right in a listening test is absolutely meaningless.

To quote: "Let me repeat, you need to completely ignore percentage of right answers. I explain this in detail in this article I wrote recently: http://www.madronadigital.com/Librar...20Testing.html. " 

Looks to me like he's still trying to right the overturned boat of the Meridian AES Conference paper.

Quote
As you are suggesting that this is very wide of the mark, I would suggest that you really need to educate the people who argue for DBTs/ABX on audio forums as they are doing ABX a disservice with their mis-information & obviously using ABX incorrectly as a result


I have never seen anybody but you argue that ABX relies only on echoic memory.  Ever.

If you bothered to look at the ABX video that I've been linking, you'd see that someone independent beat us to it.

I did quite a bit of searching on the web to see what people are saying about how to switch between sound samples during an ABX test, and I found very little actually being said. That probably means its a personal thing, and that's fine with me.

How do you listen to an ABX test?

Reply #152 –
Quote from: Arnold B. Krueger on
Quote from: greynol on
just like mzil passing your bullet-proof test?  meh.
  Mzil was rattling my cage. ;-) 
  More precisely I posted my results for the exact same reasons I posted my similar results in the AVS thread both on your keys and the AIX records test cut, "Mosaic" [A2 vs B2]. It was to document that when other people post score sheets showing they can achieve good statistical significance in differentiating A from B it doesn't necessarily mean it is because they hear a difference in the music, they may just be hearing a difference in the files, which is exactly what I did.

Lucky for you I'm on your side of the big debate so rather than toot my own horn about my excellent hearing [I'm not quite as old as you, but no spring chicken either], my excellent system [my DAC  cost all of $29, however it is  extremely, um, organic], or my extensive training*, like the shysters often do, I instead exposed to all how easy it was to show such results if you know what vulnerabilities to look for and how they will be manifested. The whole point was to negate their findings, yet at least one of them had the gall to then run off to their other forums and brag, [paraphrased] "Not only could I hear a difference but even some of the long time skeptics (meaning me) could too!"

*I actually started to take Olive's thing at one point but then realized part way through that it was NOT increasing my sensitivity to subtleties, as I had hoped it would, but rather was simply grooming me to be a more consistent reporter, so I quit.

How do you listen to an ABX test?

Reply #153 –
Quote from: Arnold B. Krueger on
According to his PM, the positive results were obtained while listening to the test tones at the end (t > 12.5 seconds), not during  the keys jangling experiment.


You can't post an ABX challenge and expect people to only listen to the subsection you tell them to, rather than choose whatever part of the file they want (especially considering our present company). You need to break it down into two files: a "Test your system for IM" file and then the actual challenge file, "Jangling Keys".

My system exhibits no tone, even at elevated levels, similar to the target test tone (I assume 4 kHz).  There are differences in the sound which are only evident at elevated levels however I don't need to ride the gain, nor clip/distort my system to hear it, so in my book I'm playing the game.

How do you listen to an ABX test?

Reply #154 –
Quote from: Arnold B. Krueger on
To quote: "Let me repeat, you need to completely ignore percentage of right answers. I explain this in detail in this article I wrote recently:

Your link doesn't work, at least for me, but then again nor do my own in recent posts which is why I've been including a back up way of finding the source. I think there is something wrong when the link gets truncated to the abbreviated form with the "..." in the middle. What does seem to still work is to highlight a particular word or phrase and turn it into a hyperlink.

How do you listen to an ABX test?

Reply #155 –
Quote from: jkeny on
In my experience the reason people "stumble over ABX" is mainly because it entails a different type of listening - it's no longer the normal, relaxed listening they are used to.

Yes, the audiophile variety of "listening" with peeking eyes, priori knowledge, status symbol worship, street cred worries etc.
IOW, very little to do with actual ear>brain listening, which is what, by definition must occur during a blind audio test. Listening and judging only by "listening with ones ears", not the convoluted audiophile version of "listening".

Quote from: jkeny on
This can probably be addressed by the test design but I seldom see it done.

Pray tell what is stopping you folks from doing this?

Quote from: jkeny on
As AJ quoted me - this is the "organic sound" that I referenced in the posts that were recently deleted.

Right. Yet you keep asking others, myself and greynol included, to "listen" for this, with no evidence that we would be able to "hear" it. You can. Just like your pal Amir insists that capable trained listeners are a must for any valid test. You meet that criteria, by your own admission. So, lets see your single blind test results of your DAC vs a 1/5th price one. Take your sweet time between switching. A week, a month, however long you feel adaptation takes in "long term listening". Just make sure there's no peeking, level mismatches, etc.
So that you are indeed "trusting your ears" and judging the sound by listening and listening only.
We await your results.

cheers,

AJ
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #156 –
So what is being demonstrated here is the complete uselessness of ABX testing - if positive results are produced there are many ways of explaining it away. If a negative result is returned there are similarly obvious ways of explaining it away.

Why would anyone use such a test except for personal use (when honesty is not usually an issue)?

How do you listen to an ABX test?

Reply #157 –
Quote
So what is being demonstrated here is the complete uselessness of ABX testing

Wrong. Any A/B test or comparison, blind or sighted, doesn't matter, would be compromised by the issue I'm keying on.

Arny can easily fix his files so my attack at a particular vulnerability is thwarted, however, by simply making two files as I previously mentioned.

How do you listen to an ABX test?

Reply #158 –
Quote from: mzil on
Quote
So what is being demonstrated here is the complete uselessness of ABX testing

Wrong. Any A/B test or comparison, blind or sighted, doesn't matter, would be compromised by the issue I'm keying on.

Arny can easily fix his files so my attack at a particular vulnerability is thwarted, however, by simply making two files as I previously mentioned.

The same honesty issue that disqualifies positive ABX results also disqualifies null results - as amply demonstrated here
 

How do you listen to an ABX test?

Reply #161 –
Quote from: ajinfla on
We await your results.

cheers,

AJ

Why would I be interested in jumping through hoops to produce test results (whether positive or negative) using a test that is so shot full of holes?

How do you listen to an ABX test?

Reply #163 –
Quote from: greynol on
Somehow we're pretending the word proctored wasn't uttered earlier?

Not ignored, just useless - what criteria do you use to judge the honesty of the proctor?
I do an ABX test & my mate proctors it - you trust the results, right?

How do you listen to an ABX test?

Reply #167 –
Quote from: jkeny on
So what is being demonstrated here is the complete uselessness of ABX testing the scientific method

Indeed science is not easy, which is I think your actual complaint, but it is still very useful if you are willing to put in the work.

How do you listen to an ABX test?

Reply #168 –
Quote from: jkeny on
Quote from: greynol on
Somehow we're pretending the word proctored wasn't uttered earlier?

Not ignored, just useless - what criteria do you use to judge the honesty of the proctor?
I do an ABX test & my mate proctors it - you trust the results, right?

As a means to bridge the division between audiophiles and objectivists, the ABX test is useless. In this sense you are right, and you go on to demonstrate the issue yourself.

However, even when there is no agreement on how to interpret the outcome of an ABX test, or whether to trust it, it is quite interesting to watch the arguments, even entertaining at times. That may not change the position of the hardliners, but it certainly benefits the more open-minded bystanders.

How do you listen to an ABX test?

Reply #169 –
Quote from: jkeny on
Why would I be interested in jumping through hoops to produce test results

To peddle your wares on the basis of repeatable, veriffiable reality, rather than organic self delusion daydreams.
No prob if you are frightened by such prospects. Most of understand that deep down inside, golden ears have no trust of their ears whatsoever, despite all the braying. 

cheers,

AJ
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #170 –
Quote from: saratoga on
Quote from: jkeny on
So what is being demonstrated here is the complete uselessness of ABX testing the scientific method

Indeed science is not easy, which is I think your actual complaint, but it is still very useful if you are willing to put in the work.
Nobody is really pretending that ABX tests conducted outside of a laboratory with expertise in psychoacoustic/neuroscience is actually science, are they?

How do you listen to an ABX test?

Reply #172 –
And it's only real science if the guys wear lab coats, obviously. Haha, c'mon jkeny, you can troll better than that.
"I hear it when I see it."

How do you listen to an ABX test?

Reply #173 –
Quote from: ajinfla on
Quote from: jkeny on
Why would I be interested in jumping through hoops to produce test results

To peddle your wares on the basis of repeatable, veriffiable reality, rather than organic self delusion daydreams.
No prob if you are frightened by such prospects. Most of understand that deep down inside, golden ears have no trust of their ears whatsoever, despite all the braying. 

cheers,

AJ

Selective quoting - the full quote is "Why would I be interested in jumping through hoops to produce test results (whether positive or negative) using a test that is so shot full of holes?"

I don't need to peddle any ware

How do you listen to an ABX test?

Reply #174 –
Quote from: xnor on
And it's only real science if the guys wear lab coats, obviously. Haha, c'mon jkeny, you can troll better than that.

You certainly demonstrate a lack of expertise in the field being examined but if you want to pretend you are doing science, go ahead while I & others chuckle at your naivete/ignorance