Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: How do you listen to an ABX test? (Read 344855 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

How do you listen to an ABX test?

Reply #200 –
It's always interesting to see how people avoid the real issue & deflect with all sorts of the usual techniques learned on forums.

How do you listen to an ABX test?

Reply #201 –
Quote from: krabapple on
Quote from: jkeny on
The point is that the test itself doesn't have the necessary controls to eliminate cheating & hence the need for proctoring.


I'd require proctoring for *you* not because ABX without positive/negative controls means 'cheating', but because I don't trust *you*.

Glad we could clear that up.



The usual rule is that any exceptional result must be repeated, and if repeated by the orginator, repeated with independent supervision or proctor.

So we have this guy over on WBF who claims to have done a single unproctored ABX test that is: "Conclusive "Proof" that higher resolution audio sounds different"

Link to Post

Now if certain people were true to their claims over here, might they have confronted that sideshow over there?

How do you listen to an ABX test?

Reply #202 –
Quote from: jkeny on
It's always interesting to see how people avoid the real issue & deflect with all sorts of the usual techniques learned on forums.



Yes, we've had a master's class in that sort of behavior over on AVS...

How do you listen to an ABX test?

Reply #203 –
Quote from: krabapple on
Quote from: jkeny on
The point is that the test itself doesn't have the necessary controls to eliminate cheating & hence the need for proctoring.


I'd require proctoring for *you* not because ABX without positive/negative controls means 'cheating', but because I don't trust *you*.

Glad we could clear that up.

Now, tell us again why yours, or anyone's , sighted evaluations of DAC sound are trustworthy?

And you would not trust any proctor I used.
Glad we could clear that up too

How do you listen to an ABX test?

Reply #204 –
Quote from: jkeny on
If you want to disregard an obvious flaw & only focus on disproving positive ABX results, I am also sorry if you do not find it acceptable.

How can one "disprove" a negative ABX result anyhow? This stuff is by its very nature not symmetrical. A negative ABX test result can be due to any number of reasons, which the test won't help you distinguish from each other. A cheating tester is only one of those reasons.

For this reason the negative result proves nothing. It means that under the circumstances of the test, with the given listeners, the differences couldn't be demonstrated as audible. It doesn't mean that there were no differences, it doesn't mean that nobody can hear the differences, it doesn't even mean that the same people would be unable to hear them in another test.

While a negative result is pretty easy to ignore, a positive isn't. If there was no flaw in the test procedure, it means that there really were audible differences. It doesn't, however, tell you what they were and what caused them.

This is a very grave asymmetry to start with. This should actually be in favor of the audiophiles, because in a symmetrical situation, a larger number of negative results would beat a smaller number of positive results. You couldn't achieve much by succeeding at an ABX test then, because you would have to outnumber the negative results to tip the balance. I am absolutely sure that you wouldn't accept that. You would consider one successful ABX test to be authoritative already.

So the situation is clearly not symmetrical, hence your whining about not giving negative results the same scrutiny is pointless at best, and dishonest at worst. Why is it necessary to give a failed ABX test such a level of scrutiny when it can and will be disregarded anyway?

You may say that we are treating negative results as being significant, contrary to what I just wrote. After all, we see the lack of valid positive results as support for our position. That is true to some extent, but it is not due to some negative results that we manufactured ourselves, it is the failures of those who claimed audibility before, which matters here. If you think that they could have cheated, or otherwise compromised the test, they would have shot themselves in the foot. I don't see why I should be reconsidering my position because of that remote possibility.

How do you listen to an ABX test?

Reply #205 –
Quote from: jkeny on
Quote from: krabapple on
Quote from: jkeny on
The point is that the test itself doesn't have the necessary controls to eliminate cheating & hence the need for proctoring.


I'd require proctoring for *you* not because ABX without positive/negative controls means 'cheating', but because I don't trust *you*.

Glad we could clear that up.

Now, tell us again why yours, or anyone's , sighted evaluations of DAC sound are trustworthy?

And you would not trust any proctor I used.
Glad we could clear that up too



Lets see if we can sumarize this. You are not going to do any ABX tests because were there any positive outcomes, we'd dismiss them.

This is what is known as a logic-tight box, because the above rhetoric is illogical, anti-scientific and self-defeating

(1) You've already precluded the possibility that you would obtain negative results and learn something from that experience.

(2) It seems to show that what's most important to you is adulation from others, not personal knowledge of the truth.

There's another possibility: You might get positive results and then some fun would begin.

Anyway, some relevant test files can be found here: High Resolution audio files for ABX-ing.

If you were right about the facts of this matter, you'd download those files, obtain positive results, and have the last laugh. It would be a great recommendation for your wunder-DACs as well.  But, you've already carted them off to the Salvation Army Store and taken the tax loss, right?

How do you listen to an ABX test?

Reply #206 –
pelmazo, don't waste your breath. We've already explained exactly that to jkeny 2 month ago. It's just bad trolling.
I have even explained to him why his suggestion to identify false negatives is flawed and easily exploited by dishonest people. If you publicly demonstrate your willful ignorance or dishonesty (like we've seen certain individuals doing in the past months) then you should expect people to reject your test results.
"I hear it when I see it."

How do you listen to an ABX test?

Reply #207 –
Quote from: pelmazo on
Quote from: jkeny on
If you want to disregard an obvious flaw & only focus on disproving positive ABX results, I am also sorry if you do not find it acceptable.

How can one "disprove" a negative ABX result anyhow? This stuff is by its very nature not symmetrical. A negative ABX test result can be due to any number of reasons, which the test won't help you distinguish from each other. A cheating tester is only one of those reasons.

For this reason the negative result proves nothing. It means that under the circumstances of the test, with the given listeners, the differences couldn't be demonstrated as audible. It doesn't mean that there were no differences, it doesn't mean that nobody can hear the differences, it doesn't even mean that the same people would be unable to hear them in another test.

While a negative result is pretty easy to ignore, a positive isn't. If there was no flaw in the test procedure, it means that there really were audible differences. It doesn't, however, tell you what they were and what caused them.

This is a very grave asymmetry to start with. This should actually be in favor of the audiophiles, because in a symmetrical situation, a larger number of negative results would beat a smaller number of positive results. You couldn't achieve much by succeeding at an ABX test then, because you would have to outnumber the negative results to tip the balance. I am absolutely sure that you wouldn't accept that. You would consider one successful ABX test to be authoritative already.

So the situation is clearly not symmetrical, hence your whining about not giving negative results the same scrutiny is pointless at best, and dishonest at worst. Why is it necessary to give a failed ABX test such a level of scrutiny when it can and will be disregarded anyway?

You may say that we are treating negative results as being significant, contrary to what I just wrote. After all, we see the lack of valid positive results as support for our position. That is true to some extent, but it is not due to some negative results that we manufactured ourselves, it is the failures of those who claimed audibility before, which matters here. If you think that they could have cheated, or otherwise compromised the test, they would have shot themselves in the foot. I don't see why I should be reconsidering my position because of that remote possibility.

OK, glad to see that you recognise that the accumulation of negative results is often used to support the claim that certain things are not audible - they are not a neutral factor nor ignored. Yes an individual null result proves nothing but an accumulation of null results is a strong indicator of inaudibility of the device/etc under test. This is not often admitted to in such discussions so it's refreshing to encounter it.

So, my position is that we don't know how "valid" these null results are - I gave the example of someone knowingly "cheating" by randomly guessing without listening. Ample evidence is already given in this thread & other such "cheating" posted by ArnyK on AVS. But there are many other situations/circumstances where null results can arise without the listener knowingly "cheating" - situations that will skew the test towards a null result. Tiredness; loss of focus; unsuitability of the environment or playback equipment to reveal small, audible differences; unsuitability of the person's hearing; disinterest/no pre-training to identify audible differences prior to test & many, many more possible reasons why a null result may be returned other than a valid null result from a genuine, valid test   

The question really is - are you interested in evaluating how many of these null results are actually valid or are you happy with the existing situation? I would love to see a genuine interest in the validity of ALL results coming from such tests & not just the positive ones being examined.

How do you listen to an ABX test?

Reply #208 –
Quote from: Arnold B. Krueger on
Quote from: jkeny on
Quote from: krabapple on
Quote from: jkeny on
The point is that the test itself doesn't have the necessary controls to eliminate cheating & hence the need for proctoring.


I'd require proctoring for *you* not because ABX without positive/negative controls means 'cheating', but because I don't trust *you*.

Glad we could clear that up.

Now, tell us again why yours, or anyone's , sighted evaluations of DAC sound are trustworthy?

And you would not trust any proctor I used.
Glad we could clear that up too



Lets see if we can sumarize this. You are not going to do any ABX tests because were there any positive outcomes, we'd dismiss them.

This is what is known as a logic-tight box, because the above rhetoric is illogical, anti-scientific and self-defeating

(1) You've already precluded the possibility that you would obtain negative results and learn something from that experience.

(2) It seems to show that what's most important to you is adulation from others, not personal knowledge of the truth.

There's another possibility: You might get positive results and then some fun would begin.

Anyway, some relevant test files can be found here: High Resolution audio files for ABX-ing.

If you were right about the facts of this matter, you'd download those files, obtain positive results, and have the last laugh. It would be a great recommendation for your wunder-DACs as well.  But, you've already carted them off to the Salvation Army Store and taken the tax loss, right?


I've asked over & over again what proctor would be accepted but got no answer. Care to answer this?
In the absence of this why would anyone waste their time doing such a test? Your phrase "You might get positive results and then some fun would begin." reveals exactly how much you are actually interested in finding truth

How do you listen to an ABX test?

Reply #209 –
Quote from: xnor on
pelmazo, don't waste your breath. We've already explained exactly that to jkeny 2 month ago. It's just bad trolling.
I have even explained to him why his suggestion to identify false negatives is flawed and easily exploited by dishonest people.
Huh? Got a link to your explanation?
Quote
If you publicly demonstrate your willful ignorance or dishonesty (like we've seen certain individuals doing in the past months) then you should expect people to reject your test results.

Who are you referring to here, exactly?

How do you listen to an ABX test?

Reply #210 –
Quote from: pelmazo on
Quote from: jkeny on
If you want to disregard an obvious flaw & only focus on disproving positive ABX results, I am also sorry if you do not find it acceptable.

How can one "disprove" a negative ABX result anyhow?


Child's play: Merely reliably and cleanly obtain a number of positive results. Preferably have a number of independent experimenters do it.

Just imagine all of the golden ears in the world ABXing away and ultimately proving the meter readers wrong. 

Quote
This stuff is by its very nature not symmetrical.


Absence of proof is not proof of absence.

Quote
A negative ABX test result can be due to any number of reasons, which the test won't help you distinguish from each other. A cheating tester is only one of those reasons.


You'd think that some people think they are the only honest people around. Some of them would like us to believe that ABX Comparators are like hen's teeth, and that only the blessed few can use them.

Quote
For this reason the negative result proves nothing. It means that under the circumstances of the test, with the given listeners, the differences couldn't be demonstrated as audible. It doesn't mean that there were no differences, it doesn't mean that nobody can hear the differences, it doesn't even mean that the same people would be unable to hear them in another test.


Exactly.

Quote
While a negative result is pretty easy to ignore, a positive isn't. If there was no flaw in the test procedure, it means that there really were audible differences. It doesn't, however, tell you what they were and what caused them.


Let's review the moving goal posts in this discussion. Once upon a time the golden ears told us that the differences were "Mind Blowing".  In those exact words.  Strangely enough they became elusive once the usual list of obvious non-audible cues were removed. Common sense suggests that since the "Mind Blowing" audible differences disappeared when the sighted cues were removed, maybe the "Mind Blowing" audible differences were due to the sighted cues. Of course correlation isn't always proof of causality, but often it is or at least it gets you looking in the right places.

How do you listen to an ABX test?

Reply #211 –
Quote from: jkeny on
I've asked over & over again what proctor would be accepted but got no answer. Care to answer this?


Sure:

JJ

Me

John Vanderkooy

Stan Lipshitz

and 100's others who are probably more conveniently physically sited

Besides, aren't you in this for the sake of knowledge and truth?  Wouldn't just knowing be enough?  It has been for me on many occasions. For example, I didn't spill the beans about the first ABX test ever done until someone did the second-dozen or more ones. Didn't want to bias them.

How do you listen to an ABX test?

Reply #212 –
Quote from: Arnold B. Krueger on
......
Let's review the moving goal posts in this discussion. Once upon a time the golden ears told us that the differences were "Mind Blowing".  In those exact words.  Strangely enough they became elusive once the usual list of obvious non-audible cues were removed. Common sense suggests that since the "Mind Blowing" audible differences disappeared when the sighted cues were removed, maybe the "Mind Blowing" audible differences were due to the sighted cues. Of course correlation isn't always proof of causality, but often it is or at least it gets you looking in the right places.

And here we have an example of a null result being used quite contrary to the neutral pretence, "doesn't prove anything" excuse used in these forums. It is very much used in this politically disingenuous way

How do you listen to an ABX test?

Reply #213 –
Quote from: Arnold B. Krueger on
Quote from: jkeny on
I've asked over & over again what proctor would be accepted but got no answer. Care to answer this?


Sure:

JJ

Me

John Vanderkooy

Stan Lipshitz

and 100's others who are probably more conveniently physically sited
Pretty much as I suspected - you need to get a grip on reality if you think any of those named people would be remotely interested in proctoring. It makes any positive ABX results impossible - just as the whole silly issue of proctoring was designed to do.

You guys have backed yourself up into a corner of reality that is untenable. All ABX tests need to be proctored, right?

Quote
Besides, aren't you in this for the sake of knowledge and truth?  Wouldn't just knowing be enough?  It has been for me on many occasions. For example, I didn't spill the beans about the first ABX test ever done until someone did the second-dozen or more ones. Didn't want to bias them.

Do you not think I haven't done my own personal blind tests & am satisfied with my personal conclusions? That is truth for me

How do you listen to an ABX test?

Reply #214 –
Quote from: jkeny on
Quote from: krabapple on
Quote from: jkeny on
The point is that the test itself doesn't have the necessary controls to eliminate cheating & hence the need for proctoring.


I'd require proctoring for *you* not because ABX without positive/negative controls means 'cheating', but because I don't trust *you*.

Glad we could clear that up.

Now, tell us again why yours, or anyone's , sighted evaluations of DAC sound are trustworthy?

And you would not trust any proctor I used.
Glad we could clear that up too



Correct.  I would get to choose or approve the proctor.  Because I don't trust you.

Now, please answer the question posed in the rest of the post:

Tell us again why yours, or anyone's , sighted evaluations of DAC sound are trustworthy?

How do you listen to an ABX test?

Reply #215 –
Quote from: jkeny on
OK, glad to see that you recognise that the accumulation of negative results is often used to support the claim that certain things are not audible - they are not a neutral factor nor ignored.



Neither are they ignored in scientific work.  Does that make science not 'neutral' either?



Quote
Yes an individual null result proves nothing but an accumulation of null results is a strong indicator of inaudibility of the device/etc under test. This is not often admitted to in such discussions so it's refreshing to encounter it.


You have no idea what you're talking about.  It is routinely 'admitted'. 



Look, let's cut to the chase around all these attempts of yours to generating FUD and reinvent the wheel:

You make and sell DACs.

You claim your DACs sound a certain way. 

How would you prove it?  Describe your method.

How do you listen to an ABX test?

Reply #216 –
Quote from: jkeny on
Pretty much as I suspected - you need to get a grip on reality if you think any of those named people would be remotely interested in proctoring. It makes any positive ABX results impossible - just as the whole silly issue of proctoring was designed to do.


Get over yourself.  It doesn't make 'any' positive ABX results impossible  -- indeed, if you had a clue, you'd know that right here on HA, there have been some highly unlikely but nevertheless 'accepted' ABX results over the years (e.g., 320 kbps mp3). 

It merely makes any highly unlikely positive ABX results *you* post, unlikely to be believed.  By me.  Others can speak for themselves.

Quote
You guys have backed yourself up into a corner of reality that is untenable. All ABX tests need to be proctored, right?


Nope.  In science one assumes good faith all the time.  But you're the one who claims it's easy to cheat, right?

Quote
Do you not think I haven't done my own personal blind tests & am satisfied with my personal conclusions? That is truth for me


But not 'just 'for you', eh?  You claim your DACs will  sound different/better...presumably not just to you.

How do you listen to an ABX test?

Reply #217 –
Quote from: jkeny on
The question really is - are you interested in evaluating how many of these null results are actually valid or are you happy with the existing situation? I would love to see a genuine interest in the validity of ALL results coming from such tests & not just the positive ones being examined.

You only seem to read half of what I wrote. No, I'm not interested in evaluating that, because there is no such thing as a null result being valid or invalid to start with. The mere concept is bunk. Snap out of it!

For example, say I complete an ABX test with a null result, and my friend completes the same test with a success. Does that make my test invalid or not? On what grounds do you decide that? Assume I didn't cheat or play games with the test. Isn't it perfectly normal for such a test that some people fail and others succeed, even when nothing is wrong with the test? Could be due to differences in hearing ability, in training, in form, in patience, in luck ... You are listing them yourself, yet you don't draw the obvious conclusion!

The only thing you can say is that the test succeeded or didn't succeed. If it didn't, you won't know why. You'd have to design another test to find that out if you wanted to know more. Only if it succeeded, the question arises whether it succeeded because of a condition that made the test invalid.

Quote
And here we have an example of a null result being used quite contrary to the neutral pretence, "doesn't prove anything" excuse used in these forums. It is very much used in this politically disingenuous way

If a null result doesn't prove anything, the default position until proven otherwise has to be to reject the claim. That a series of null results bolsters this position may look disingenious to you, but in fact it changes nothing. A claim that has to be rejected anyway can - strictly speaking - not be rejected even more, nor does it need to be rejected more. The increased confidence in the rejection is purely psychological, yet it is perfectly reasonable. If it doesn't please you, you are free to ignore it. The rejection remains anyway.

Quote from: xnor link=msg=0 date=
It's just bad trolling.

Looks like it.

How do you listen to an ABX test?

Reply #218 –
Quote from: krabapple on
Quote from: jkeny on
OK, glad to see that you recognise that the accumulation of negative results is often used to support the claim that certain things are not audible - they are not a neutral factor nor ignored.



Neither are they ignored in scientific work.  Does that make science not 'neutral' either?



Quote
Yes an individual null result proves nothing but an accumulation of null results is a strong indicator of inaudibility of the device/etc under test. This is not often admitted to in such discussions so it's refreshing to encounter it.


You have no idea what you're talking about.  It is routinely 'admitted'. 
OK, good, my statement is wrong & it is admitted that null results are of importance. Therefore the validity of these null results must be of importance - after all you don't want to be be strongly swayed by invalid results, do you?

So then proctoring is needed, right?


Quote
Look, let's cut to the chase around all these attempts of yours to generating FUD and reinvent the wheel:

You make and sell DACs.

You claim your DACs sound a certain way. 

How would you prove it?  Describe your method.


It's not about my DACs - it's about how ABX tests are done
I will never be able to "prove" that my DAcs sound the way I describe - it's up to individual customers to decide this in whatever way they see fit - sighted/blind listening. They have 30 days return with no restocking fees so it's their personal tests that count not a case of "proving" or "persuading" anybody of anything.

When it comes to tests that are not personal tests & used to "prove" or "strongly indicate" something, then a whole raft of other considerations need addressing

How do you listen to an ABX test?

Reply #219 –
Quote
Quote from: jkeny on
Pretty much as I suspected - you need to get a grip on reality if you think any of those named people would be remotely interested in proctoring. It makes any positive ABX results impossible - just as the whole silly issue of proctoring was designed to do.


Get over yourself.  It doesn't make 'any' positive ABX results impossible  -- indeed, if you had a clue, you'd know that right here on HA, there have been some highly unlikely but nevertheless 'accepted' ABX results over the years (e.g., 320 kbps mp3). 

It merely makes any highly unlikely positive ABX results *you* post, unlikely to be believed.  By me.  Others can speak for themselves.
So, as I said the test is useless as a test other than a personal test - it leads nowhere - anyone can decide that they don't trust you & demand proctoring - it's complete rubbish. Logically, if you insist on proctoring then it has to be done for all ABX tests!!

Quote
Quote
You guys have backed yourself up into a corner of reality that is untenable. All ABX tests need to be proctored, right?


Nope.  In science one assumes good faith all the time.  But you're the one who claims it's easy to cheat, right?
Nope, I'm not the one that raised the requirement for proctoring - it was first raise as the final "excuse" to deny Amir's results. Cheating was then demonstrated by ArnyK in his "false" null ABX results. It was demonstrated here by Mzil posting a number of ABX null results with his random guessing

How do you listen to an ABX test?

Reply #220 –
Quote from: pelmazo on
Quote from: jkeny on
The question really is - are you interested in evaluating how many of these null results are actually valid or are you happy with the existing situation? I would love to see a genuine interest in the validity of ALL results coming from such tests & not just the positive ones being examined.

You only seem to read half of what I wrote. No, I'm not interested in evaluating that, because there is no such thing as a null result being valid or invalid to start with. The mere concept is bunk. Snap out of it!

For example, say I complete an ABX test with a null result, and my friend completes the same test with a success. Does that make my test invalid or not? On what grounds do you decide that? Assume I didn't cheat or play games with the test. Isn't it perfectly normal for such a test that some people fail and others succeed, even when nothing is wrong with the test? Could be due to differences in hearing ability, in training, in form, in patience, in luck ... You are listing them yourself, yet you don't draw the obvious conclusion!

The only thing you can say is that the test succeeded or didn't succeed. If it didn't, you won't know why. You'd have to design another test to find that out if you wanted to know more. Only if it succeeded, the question arises whether it succeeded because of a condition that made the test invalid.
That would be fine if all null results were disregarded & considered of no significance but this isn't the case as you & krabapple have admitted - they form part of the body of "evidence".  No, "Results" presuppose that a real "test" had actually taken place. In the ABX null results posted here by mzil, where he just randomly guessed - do you consider that he "took a test"? Do you consider his "results" have any meaning or should they be eliminated from the "accumulated body of evidence"?

If I sat a monkey down in front of the keyboard would I most likely get the same null results? Would you count the monkey's results among the null results?
Would you consider the results produced by a deaf person as a valid test? What about someone who demonstrated a hearing impairment in the audible area being tested? What about someone who has demonstrated that they are pre-biased to not hearing any differences? What about someone who is so tired that they aren't focussed? What about playback equipment that is unsuitable for revealing differences? Do I need to go on?

In all properly designed scientific tests, controls are used to eliminate conditions that disqualify the results from being counted among the valid results.

Quote
Quote
And here we have an example of a null result being used quite contrary to the neutral pretence, "doesn't prove anything" excuse used in these forums. It is very much used in this politically disingenuous way

If a null result doesn't prove anything, the default position until proven otherwise has to be to reject the claim. That a series of null results bolsters this position may look disingenious to you, but in fact it changes nothing. A claim that has to be rejected anyway can - strictly speaking - not be rejected even more, nor does it need to be rejected more. The increased confidence in the rejection is purely psychological, yet it is perfectly reasonable. If it doesn't please you, you are free to ignore it. The rejection remains anyway.
Yes, & the important word here is "result" - you have to decide what is a result & what should be eliminated from "results". In your statements you are including everything into results even the deaf monkey's "results"

How do you listen to an ABX test?

Reply #221 –
What we have here is a great example of experimenter's bias or research bias where the test is skewed towards a particular result & any attempt at pointing out how it might be improved are rejected.

You guys cite biasing so much that I'm sure you know about this particular bias - so why the rabid rejection of examining the ABX test?

How do you listen to an ABX test?

Reply #222 –
Quote from: jkeny on
No, results presuppose that a "test" was actually taken. In the ABX null results posted here by mzil, where he just randomly guessed - do you consider that he "took a test" & delivered "results"?

It doesn't matter. Noone except him could tell the difference. That's why your whole idea of an invalid null result is bunk.

And let me add this: The fact that it doesn't matter is an important quality of such a test. It speaks for the ABX test method, and is a major factor in its usefulness.

Quote
If I sat a monkey down in front of the keyboard would I most likely get the same null results? Would you count the monkey's results among the null results?

I am not supposed to judge the monkey. If he produces a null result, I count it as a null result. If he produces a result that deviates significantly from chance, I count it accordingly. If I did anything else I'd be rigging the test.

Quote
Would you consider the results produced by a deaf person as a valid test? What about someone who demonstrated a hearing impairment in the audible area being tested? What about someone who has demonstrated that they are pre-biased to not hearing any differences? What about someone who is so tired that they aren't focussed? What about playback equipment that is unsuitable for revealing differences? Do I need to go on?

No, you needn't go on. I would count all of them as valid results. Doing anything else would put my own judgement above their results. I would effectively override their test results, thereby making the test invalid. Any test is invalid if the test administrator is allowed to override the test results of selected participants. Isn't that abundantly clear?

Now, it is true that a hearing test conducted with monkeys or deaf people might be regarded as pointless. That is an unfortunate consequence of a poor test design. If you aren't interested in the hearing abilities of monkeys or deaf people, you should exclude them from the test before the start. Once they are in, they are in - this doesn't make the test invalid. It may merely make it useless, depending on what the question was. This is so by design, I have to emphasize it again! It is not a fault or deficiency of ABX, quite the opposite, it is a major factor of its usefulness.

Quote
Yes, & the important word here is "result" - you have to decide what is a result & what should be eliminated from "results". In your statements you are including everything into results even the deaf monkeys "results"

Yes, that is quite deliberately so. You are trying to ridicule my position, but you miss that it isn't ridiculous at all. It is the opposite: Your position puts the designer/administrator of the test into a position where he can manipulate the criteria for result acceptance to his liking, potentially even after the fact. That's what I would call invalid!

And you wonder why other people don't trust you as a test designer/administrator? Amusing!
 

How do you listen to an ABX test?

Reply #223 –
Quote from: jkeny on
All ABX tests need to be proctored, right?

Can you cite some AES ones that aren't?
Yes, generally speaking they should be, especially when done by known shysters and those with strong pecuniary interests in the audio fashion jewelry business.

Quote from: jkeny on
Do you not think I haven't done my own personal blind tests & am satisfied with my personal conclusions? That is truth for me

Yes, as has your pal. We don't care about those (outside of the tremendous entertainment factor  ), or your own sub 9 second 100m dashes in the back yard, or whether you've bent spoons in front of your wife and friends.
We need more than photoshopped unicorn pics around here. We also understand that there are many who don't and will accept your daydreams as concurrent reality with theirs. But this isn't their hangout.

Quote from: jkeny on
I will never be able to "prove" that my DAcs sound the way I describe

We know.
Blind tests are great for revealing real audio differences. They are worthless for apparitions. So the apparition believers reject them and will seek to discredit them.
Especially those with strong pecuniary interests.

cheers,

AJ
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #224 –
I think by his [jkeny] logic, if some student were to fill out a true/false test in school without reading the questions and simply checking all the "true" boxes, this would prove that true/false tests "don't work" and that the entire test methodology should be scrapped.