Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: How do you listen to an ABX test? (Read 343910 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

How do you listen to an ABX test?

Reply #1500
1) Avrev.com: Can you tell us a bit about your background, and how you got into the business of designing and selling USB DACs?

John Kenny: My professional background doesn't involve audio....
...Always in search of better (fake data) "sound", I have since moved to using a different USB receiver board & a TI/BB DAC chip which (fake data) sounds even better than the Sabre DAC. The fake data sound I'm attuned to is an organic, full-bodied sound with all the detail intact which conveys a great sound stage & excellent instrument timbre. Almost like the sound Santa makes landing on my roof.

I suggest you read this short piece "Why faking data is bad" - it might give you some insights into your logic - then again it probably won't

 
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #1501
No offence taken but I & others have done numerous side by side comparisons of stock unit to modified unit - were they DBT, no - there is absolutely no need, the difference in sound is so glaringly obvious & noticeable from the first couple of notes. No offence but I already have given this proof with links to the reviews. Have you read any? Anecdotal fake data, yes & not worth a damn according to those who have never heard the unit - say this to one of the people who have a unit & they will laugh at your stupidity in requiring DBT.

Yes, I get better, more comprehensive results with longer term "listening"(chronic peeking)

Yep, I've no doubt that sighted tests are prone to false positives

As I said, I personally check any I am interested in - so I don't accept other's sighted reports.

As I said if I'm interested, I check it myself. So I have verified to my own satisfaction that sighted results can be correct.

I think it is well known & accepted that sighted listening can be prone to false positives - I don't know to what extent - you say "overflowing" but I don't personally experience that.

I suggest you read this short piece "Why faking data is bad" - it might give you some insights into your logic - then again it probably won't

     
Loudspeaker manufacturer

 

How do you listen to an ABX test?

Reply #1502
- Arny tries to claim that there are sufficient controls in ABX because his test results were exposed as fraud - put simply, his results revealed his stupidity - if he had taken just a bit longer between each random guess nobody would have been any wiser about this fraud (he certainly wouldn't have admitted it)


Keny thus again denies the proof of success of a test control that he has cited dozens of times.  His new supposition is starts out with "if". There are a milliion if's that could be raised but their existence is not proof.

Here is reliable and incontrovertible proof is that JKeny is a habitual liar: How many times has Keny posted that ABX tests have nocontrols related to false negatives?

Google shows that Kenny posted this phrase dozens of times:

"Typical ABX/blind tests that don't have false negatives controls are not to be trusted"

Then Keny remembers what he has known for months,  that typical ABX tests do have false negative controls. He tries to use selective quoting to falsely take advantage of them in a vain attempt to humiliate me, and then goes back to falsely claiming:

"Typical ABX/blind tests that don't have false negatives controls are not to be trusted"

Google finds about 74 similar results!

The ABX timing log isn't a control for false negatives - it simply reveals anybody who is stupid enough to hit the same button again & again, quickly, while at the same time pretending to do an ABX test. It is very seldom such stupidity is revealed so, Arny, I wouldn't cite this as a control unless you believe there are many people like yourself who try to fake it & this is the only source of false negatives?

How do you listen to an ABX test?

Reply #1503
Yep & as predicted, AJ, never fails to perform
Dance, AJ, dance

How do you listen to an ABX test?

Reply #1504
Afair Arnold B. Krueger was not guessing but instead intentionally pressing "A" continuously as answer; am i mistaken?

It has no way of determining if any null result is valid.


I still do not get what you think is a "valid" conclusion that the experiment could not reject the null.


An experiment will (aside from chance) only be able to reject the null, if it is valid (means that it really tests what it pretends to test) and if the power of the test is sufficient to avoid type II errors.


That was not really an answer to the question (except what you write in parentheses highlight the issue with sighted tests - they do not test the sound). What I was trying to get jkeny into elaborating on, is whether he actually agrees that it is "valid" to keep the null until data tells otherwise.

Thus far jkeny has dodged even the most basic concepts, so I doubt I will ever get to what he means by "control" for
  • .

How do you listen to an ABX test?

Reply #1505
Yes, Jakob, As I say the more they post the more revealing it becomes - the basics are still a mystery to some"
Quote
"An experiment will (aside from chance) only be able to reject the null, if it is valid (means that it really tests what it pretends to test) and if the power of the test is sufficient to avoid type II errors."

How do you listen to an ABX test?

Reply #1506
The ABX timing log isn't a control for false negatives - it simply reveals anybody who is stupid enough to hit the same button again & again, quickly, while at the same time pretending to do an ABX test. It is very seldom such stupidity is revealed so, Arny, I wouldn't cite this as a control unless you believe there are many people like yourself who try to fake it & this is the only source of false negatives?

I've already shown you to be wrong over 3 months ago here.
You must be really desperate.


@mods: I'm seriously wondering why this stupidity has been tolerated for so long in this thread. You locked the other one down pretty quickly after jkeny started ... being himself.

"I hear it when I see it."

How do you listen to an ABX test?

Reply #1507
Afair Arnold B. Krueger was not guessing but instead intentionally pressing "A" continuously as answer; am i mistaken?

It has no way of determining if any null result is valid.


I still do not get what you think is a "valid" conclusion that the experiment could not reject the null.


An experiment will (aside from chance) only be able to reject the null, if it is valid (means that it really tests what it pretends to test) and if the power of the test is sufficient to avoid type II errors.


That was not really an answer to the question (except what you write in parentheses highlight the issue with sighted tests - they do not test the sound). What I was trying to get jkeny into elaborating on, is whether he actually agrees that it is "valid" to keep the null until data tells otherwise.

Thus far jkeny has dodged even the most basic concepts, so I doubt I will ever get to what he means by "control" for
  • .

What Jakob is telling you is that a null (or any result) can only be the result of a "valid" test - it can't be the outcome of a test where you sit 100 monkeys in front of a keyboard who are trained to continuously hit the A button. If you don't understand this, you fail to understand the basics of what a "test" means

How do you listen to an ABX test?

Reply #1508
No offence taken but I & others have done numerous side by side comparisons of stock unit to modified unit - were they DBT, no - there is absolutely no need, the difference in sound is so glaringly obvious & noticeable from the first couple of notes. No offence but I already have given this proof with links to the reviews. Have you read any? Anecdotal fake data, yes & not worth a damn according to those who have never heard the unit - say this to one of the people who have a unit & they will laugh at your stupidity in requiring DBT.

Yes, I get better, more comprehensive results with longer term "listening"(chronic peeking)

As I said, I personally "check" any I am interested in - so I don't accept other's sighted reports.

As I said if I'm interested, I delude myself. So I have verified to my own satisfaction that sighted results can be correct.

you fail to understand the basics of what a "test" means

More, more!!
Thanks John
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #1509
How does any of this preclude asking a person for ABX data, to support his claim that he heard a difference?


Dance, dance


John, how does any of this preclude asking a person for ABX data, to support his claim that he heard a difference?

Dance, dance


John??
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #1510
What Jakob is telling you is that a null (or any result) can only be the result of a "valid" test - it can't be the outcome of a test where you sit 100 monkeys in front of a keyboard who are trained to continuously hit the A button. If you don't understand this, you fail to understand the basics of what a "test" means

What about one monkey "confirming" what 3 other monkeys told him he would "hear", daydreaming?
Test?
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #1511
John, if you are afraid to do a blind test of any audiophile daydream-belief, such as your DAC sounding "organic" , does that make blind tests:
1) Frightening to kids?
2) Prone to preponderant negative results-cheating?
3) "I'm going to look like a damn fool" stressful?
4) Stupidity?
Or..??
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #1512
The ABX timing log isn't a control for false negatives - it simply reveals anybody who is stupid enough to hit the same button again & again, quickly, while at the same time pretending to do an ABX test. It is very seldom such stupidity is revealed so, Arny, I wouldn't cite this as a control unless you believe there are many people like yourself who try to fake it & this is the only source of false negatives?

I've already shown you to be wrong over 3 months ago here.
You must be really desperate.


@mods: I'm seriously wondering why this stupidity has been tolerated for so long in this thread. You locked the other one down pretty quickly after jkeny started ... being himself.

So what your link shows is that you can use a "tell" (a timing click) to spot differences between files & you can do this in 1 or 2 seconds for each trial.
Your point is that you can also avoid doing a genuine listening test just like Arny except you are using two files with an easy "tell" to differentiate them so you don't get a null result, like Arny did with his random guessing. Brilliant revelations!

That's ridiculous - just proves just how little you know about the basics of what a listening test means. You tried first to pass off a listening test of two different songs to show trials could be done in 1 or 2 secs & get it right - Duh!!

As I say, keep posting - it is very revealing!

How do you listen to an ABX test?

Reply #1513
Your point is that you can also avoid doing a genuine listening test

Yes! At all costs!
Don't worry John, we adults know why you do.
Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #1514
What Jakob is telling you is that a null (or any result) can only be the result of a "valid" test - it can't be the outcome of a test where you sit 100 monkeys in front of a keyboard who are trained to continuously hit the A button.


That's ludicrous, but to apply your own grotesquely flawed logic Kenny, please provide accurate estimates of the false positives and false negatives that occur during your reference standard - casual audiophile sighted evaluations.

Since no way are you going to touch this question Keny, so I'll give you an accurate scientific answer:

The percentage of false positives and false negatives that arise during the average casual audiophile sighted evaluations is exactly 50% each.

How did I arrive at that?

The typical audiophile sighted evaluation involves an audible difference that if it were properly presented to the listener would generate true negatives. They all involve what are basically below threshold differences, especially the ones involving DACs. Given all of the confounding influences such as sighted listening, absent effective level matching, absent time synchronization of the music, slow switching,, etc. the expected results are the same as random guessing.  If you apply random guessing, the results will be about half false negatives and about half false positives.

Quote
If you don't understand this, you fail to understand the basics of what a "test" means


Well that, as well. The most important characteristic of a test is comparison with a known reference standard. Since audiophile sighted evaluations generally do not involve comparisons with a known reference standard, they are not properly called tests. That's why I call them evaluations, since it would be lying like a, well like a,  well like a JKeny to call them tests!

How do you listen to an ABX test?

Reply #1515
So what your link shows is that you can use a "tell" (a timing click) to spot differences between files & you can do this in 1 or 2 seconds for each trial.
Your point is that you can also avoid doing a genuine listening test just like Arny except you are using two files with an easy "tell" to differentiate them so you don't get a null result, like Arny did with his random guessing. Brilliant revelations!

The complete lack of any logical thinking has already been exposed here.


That's ridiculous - just proves just how little you know about the basics of what a listening test means. You tried first to pass off a listening test of two different songs to show trials could be done in 1 or 2 secs & get it right - Duh!!

As I say, keep posting - it is very revealing!

You honestly disgust me.

I will start reporting you if you continue going down this road. We've even had to explain what ABX is to you, among other basics ...
"I hear it when I see it."

How do you listen to an ABX test?

Reply #1516
So, Arny, let's examine your logic - you want false positive & false negative statistics for something you don't consider a test - sighted evaluations.

You consider these statistics important for a test but don't have false negative statistics for ABX tests.

Keep posting, all, it's very revealing

How do you listen to an ABX test?

Reply #1517
So, Arny, let's examine your logic

John, how does Arny's one alleged "Cheating Negative" explain you being a chronic peek-o-phile?
Can you explain the logic of how Arny's very recently uncovered nefarious plot, made you say this 5yrs ago:
No offence taken but I & others have done numerous side by side comparisons of stock unit to modified unit - were they DBT, no - there is absolutely no need, the difference in sound is so glaringly obvious & noticeable from the first couple of notes. No offence but I already have given this proof with links to the reviews. Have you read any? Anecdotal fake data, yes & not worth a damn according to those who have never heard the unit - say this to one of the people who have a unit & they will laugh at your stupidity in requiring DBT.

You didn't seem overly concerned with "Cheating Negatives", in what is now "ABX", back then?


Loudspeaker manufacturer

How do you listen to an ABX test?

Reply #1518
So, Arny, let's examine your logic - you want false positive & false negative statistics for something you don't consider a test - sighted evaluations.


Well JKeny, You've educated me to demand a test for which both false positive and false negative statistics are available.  Courtesy of several academic papers, we have them for ABX tests.  Where are the same statistics for sighted evaluations?

You seem to be the big advocate for sighted evaluations around here, so where are your statistics?

You've been politely asked for them twice now, and as predicted, you have been a complete and total no show.

So, why is important for ABX tests to have these statistics available (which we do have in hand)  but not sighted evaluations?

Or, are the estimated false positive and false negative statistics that I've posted for sighted evaluations correct?


How do you listen to an ABX test?

Reply #1519
@mods: I'm seriously wondering why this stupidity has been tolerated for so long in this thread. You locked the other one down pretty quickly after jkeny started ... being himself.

My guess is that the mods wanted to let more jkeny's potential customers watching this thread in order to ruin his business.

How do you listen to an ABX test?

Reply #1520
The active parties still appear interested in continuing the discussion in the vein that has manifested; and they are free to stop posting at any time.

How do you listen to an ABX test?

Reply #1521
<snip>

That was not really an answer to the question (except what you write in parentheses highlight the issue with sighted tests - they do not test the sound).


Sorry, at that point i did not understand your question. What you expressed in parentheses is misleading, because you simply don´t know if a "sighted test" tests the sound. You might assume that it did not, but you don´t really know. As stated before the problem with a sighted test is, that you can´t show its validity.


Quote
What I was trying to get jkeny into elaborating on, is whether he actually agrees that it is "valid" to keep the null until data tells otherwise.


That is the very irritating part of the discussion; jkeny has valid points and "keeping the null" would be much more justified if these points were really included in tests. As said before, as very similar points were discussed in Meilgaards book, which is referenced in the hydrogenaudio wiki, i don´t really understand why they are still so controversial.

"Keeping the null" is normally not a valid consideration, because, as said before, the "null " will not really be tested.
Otoh the null hypothesis reflects a reasonable point of view, as it represents a hypothesis based on known data, but it is still a hypothesis.
So it is more a valid consideration to express that you don´t believe in an audible difference (don´t think that something is audible) until experimental results show the contrary.

Quote
Thus far jkeny has dodged even the most basic concepts, so I doubt I will ever get to what he means by "control" for
  • .


Maybe it would be much more productive to refrain from all this personal stuff to get the interesting points settled.

How do you listen to an ABX test?

Reply #1522
The point that you seem to have missed is that everything that jkeny has said here is just a ruse. He comes here claiming that ABX test results are invalid due to some claimed deficiencies, and that he has ideas on how to overcome said deficiencies.

The truth is that the last thing that jkeny wants is for those ideas to get implemented so that his claim of "false negatives" is disproved and he no longer can claim that the reason that ABX testing does not support his claims of audible superiority of his products is that ABX testing is flawed.

How do you listen to an ABX test?

Reply #1523
<snip>

Review thread for all the times it's been said that negative results aren't particularly interesting. If you would adopt the orientation presented here, your problems would go away -- That simple.

Inattentional deafness is not possible to quantify, so you allow for it.


As transparency is one of the goals of course "negative results" are interesting.
If you follow the discussions in forums, you´ll certainly notice that the negative results show up in a number of arguments; mostly along the lines , that numerous failed tests showed that......

The inventors of the ABC/HR developed a quite ingenious way to deal with a lot of these points. Allowing for something without being able to quantify the impact is questionable.




Transparency isn't always a goal. For TOS 8:

  8. All members that put forth a statement concerning subjective sound quality, must -- to the best of their ability -- provide objective support for their claims. Acceptable means of support are double blind listening tests (ABX or ABC/HR) demonstrating that the member can discern a difference perceptually, together with a test sample to allow
    others to reproduce their findings. Graphs, non-blind listening tests, waveform difference comparisons, and so on, are not acceptable means of providing support.

It's simply a claim of audibility.

How do you listen to an ABX test?

Reply #1524
The point that you seem to have missed is that everything that jkeny has said here is just a ruse.


Jakob2 hasn't missed anything. Despite all the fancy talk, he's a daydream believer also. Check out his posting history on Diyaudio under that name.

cheers,

AJ
Loudspeaker manufacturer