Skip to main content
Topic: History and Accreditation of ABX Testing/Invention? (Read 20244 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

History and Accreditation of ABX Testing/Invention?

Reply #25
History and Accreditation of ABX Testing/Invention

For a while I have been curious as to origins of ABX testing.  Reading forum posts, it would indicate Arny invented the scheme.  You can see that from the post in the parallel thread and many quotes in AVS Forum:
However the ASA was publishing a great many papers based on the JASA version of ABX long before the AES even mentioned the AES version of ABX.  People who read journals and don't just drop their names know such things.  They also know the nature of the differences between the two. One of the items in the committee review of Clark's AES ABX paper included clarifying these things because enough people in the AES knew about the ASA version of ABX whcih came much earlier.  So did we.  BTW the letters ABX in the AES were originally a play on my initials.




As is his usual habit Amir did not reproduce about half of the relevant evidence in this matter - the part that doesn't fit with his agenda.

Here are some relevant quotes from Clark's 1982 JAES ABX paper:

"
Listening tests used to evaluate audio equipment
seldom be considered scientific tests. For example,
extraneous factors which could influence the listener's
decision are not eliminated or held constant.
A common failing is lack of a double-blind procedure because
of its inconvenience.

When scientific tests have been performed, listeners'
thresholds have appeared to be poorer by
orders of magnitude compared to casual tests.
It has been been argued that the methods and equipment used in
the scientific test have inhibited the listener's discriminatory ability.
"

Obviously, this argument as to whether adding science inhibits the listener's ability to hear differences has gone on for another 32 or more years, but the common technique used by the anti-science contingent was exactly the one
use by Amir and the Meridian crew, which is to pretend that the 1950 version of ABX is the only version of ABX. Amir must have this paper at his disposal - he's quoted from it on AVS.

To continue from the 1982 ABX JAES paper:

"
A system of double-blind comparison testing, referred to as the A/B/X method, is described.
This system consists of:
1) Techniques to maximize discriminatory ability and the validity (trustworthiness) that it can provide.
2) Procedures and standards for maintaining validity
3) Commercially available double-blind test equipment.

It is the intent of this system to make practical the widespread use of scientific double-blind listening tests.
"

"
The most common type of listening test incorporating
a control is the A/B test. Two components, A and
B, are switched into the audio chain in turn so that a
comparison can be made. One component can be considered
the factor under study and the other the comtrol.

One component is preferably a wire bypass so that any
differences heard can be presumed to be distortion
added by the real component.
"

"The author's first experience with double-blind audibility testing
was as a member of the SMWTMS Audio
Club in early 1977. A button was provided which would
select at random component A or B. Identifying one of
these, the X component was greatly hampered by not
having the known A and B available for reference. This
was corrected by using three interlocked push buttons,
A, B, and X. Once an X was selected, it would remain
that particular A or B until it was decided to move on
to another random selection.
"

The above paragraph is just an anecdotal version of the list of differences between ABX1950 and ABX1982  that I have already contributed to this thread, but contains the same basic information either explicitly or by implication.

History and Accreditation of ABX Testing/Invention?

Reply #26
Good morning Arny and other members.  I was in the middle of reading Arny's last post and darn Windows decided to reboot without warning . I came back and it is gone???  It was Arny addressing Steven (Krab).  Did you delete it Arny?

Anyway, I was still trying to stitch this story together and landed unintentionally on another AVS Forum set of posts discussing this very topic.  So the interest to clear up the history here is shared more broadly than me. 

[quote author=arny on AVS link=msg=0 date=]
[quote author=kodi41 link=msg=0 date=]
this is for arnyk, you stated, and I quote:Quote: I invented ABX and was among the organizers and participants in many of those tests that are in the archives.

IF this be the case- then you can easily describe what equipment was used- brand, model numbers and the source player(s) pre-amp and speakers that were used, as well at what power level this testing was performed at. As well, how long the time of listening sessions were. And as well what type of music was used.
[/quote]
I did the first ABX test in late April, 1977. No, it is not easy for me to describe what equipment was used- brand, model numbers and the source player(s) pre-amp and speakers that were used, as well at what power level this testing was performed at. As well, how long the time of listening sessions were. And as well what type of music was used.[/quote]

Arny, help us out please.  How could your recollection of something this important in your life as to have built an entire persona around it on the Internet be so poor?  I often hear you tell stories about old gear you had, their performance, design, etc.  But here, no recollection at all?  No way to document what led to your strong belief in efficacy of said invention?  Nothing at all?

Amir
Retired Technology Insider
Founder, AudioScienceReview.com

History and Accreditation of ABX Testing/Invention?

Reply #27
Good morning Arny and other members.  I was in the middle of reading Arny's last post and darn Windows decided to reboot without warning . I came back and it is gone???  It was Arny addressing Steven (Krab).  Did you delete it Arny?

Anyway, I was still trying to stitch this story together and landed unintentionally on another AVS Forum set of posts discussing this very topic.  So the interest to clear up the history here is shared more broadly than me. 

[quote author=arny on AVS link=msg=0 date=]
[quote author=kodi41 link=msg=0 date=]
this is for arnyk, you stated, and I quote:Quote: I invented ABX and was among the organizers and participants in many of those tests that are in the archives.

IF this be the case- then you can easily describe what equipment was used- brand, model numbers and the source player(s) pre-amp and speakers that were used, as well at what power level this testing was performed at. As well, how long the time of listening sessions were. And as well what type of music was used.

I did the first ABX test in late April, 1977. No, it is not easy for me to describe what equipment was used- brand, model numbers and the source player(s) pre-amp and speakers that were used, as well at what power level this testing was performed at. As well, how long the time of listening sessions were. And as well what type of music was used.[/quote]

Arny, help us out please.  How could your recollection of something this important in your life as to have built an entire persona around it on the Internet be so poor?  I often hear you tell stories about old gear you had, their performance, design, etc.  But here, no recollection at all?  No way to document what led to your strong belief in efficacy of said invention?  Nothing at all?
[/quote]

The first big mistake is hanging so much weight on what are small details.  Amir as I understand it, you were alive and kicking in late April 1977. Why don't you accomplish that you fault me for not doing - list out the equipment you used- brand, model numbers and the source player(s) pre-amp and speakers that were used, as well at what power level this listening was performed at. As well, how long the time of listening sessions were. And as well what musical tracks you were listening to on that day.

That all said with some thought I come up with the following:

The amplifiers being compared in my first ABX test were the amplifiers in my Heath AR1500 versus my Dyna 400. The preamp was the preamp section of the AR1500. The turntable was a Thorens TD125 with a SME 3009 tone arm and V15 cartrdige. The speakers were Ohm Fs.  The power had to be fairly limited because pushing the AR1500 even just close to clipping (60 wpc) would take out the fast blow speaker fuses in the Ohm Fs.  One of the musical tracks I played was the Sheffied D2D "King James Version".  There were full-track familiarization sessions and shorter test sessions that were scored.

History and Accreditation of ABX Testing/Invention?

Reply #28
The first big mistake is hanging so much weight on what are small details.  Amir as I understand it, you were alive and kicking in late April 1977. Why don't you accomplish that you fault me for not doing - list out the equipment you used- brand, model numbers and the source player(s) pre-amp and speakers that were used, as well at what power level this listening was performed at. As well, how long the time of listening sessions were. And as well what musical tracks you were listening to on that day.

Used for what Arny?  I have said nothing about an invention.

But sure, I had built a kit amplifier then.  I still remember the board vividly.  It was only 10 watts or so.  I hooked them up to our (my brother and I's) Pioneer speakers.  They were this beautiful series:

.

I remember all of this very distinctly.  How could a man forget such detail about his beloved hobby putting aside it being an invention? 

Quote
The amplifiers being compared in my first ABX test were the amplifiers in my Heath AR1500 versus my Dyna 400. The preamp was the preamp section of the AR1500. The turntable was a Thorens TD125 with a SME 3009 tone arm and V15 cartrdige. The speakers were Ohm Fs.  The power had to be fairly limited because pushing the AR1500 even just close to clipping (60 wpc) would take out the fast blow speaker fuses in the Ohm Fs.  One of the musical tracks I played was the Sheffied D2D "King James Version".  There were full-track familiarization sessions and shorter test sessions that were scored.

Great!  We accomplished something.  How come you said you didn't recall it in the other thread?  Is this the first time you have ever listed this gear?
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

History and Accreditation of ABX Testing/Invention?

Reply #29
Arny,

I'll just say, you seem constitutionally unable to RESIST rising to Amir's bait, no matter how obvious or trivial it is.    Have you any sense of how many threads on how many forums have devolved mainly into back and forth between just the two of you, often rehashing points and posts made so long ago that no one else remembers them?  I do have a sense of that.

Quote
Quote
You and David Clarke *did* take A/B/X comparison (or re-invent it yourselves) and make it something less academic, and more directly meaningful and useful, to home audio.  And made it part of the parlance of the audio hobby.  There is honor enough in that.


I had nothing to do with David Clarke. The man you are probably thinking of is David Clark, no E.  BTW there is a document somewhere signed by David Clark and Bernhard Muller and 3 other witnesses that agrees that I invented the Improved ABX test that among other things, Clark wrote up for the JAES.


(sigh).  OK, Clark with no 'e' and OK, Improved ABX test' then (which implies, to me, that ABX has already been 'invented', but: whatever.  Improvement matters too.) 

These strike me as  truly insignificant details with respect to a substantive 'critique' of ABX comparison, which is what I thought Amir was promising us over on the other thread.

So, I'm out of this thread.  Watching you two fight like rabid cats in a bag over something this peripheral, just isn't that informative or even entertaining.

History and Accreditation of ABX Testing/Invention?

Reply #30
Arny, help us out please.  How could your recollection of something this important in your life as to have built an entire persona around it on the Internet be so poor?  I often hear you tell stories about old gear you had, their performance, design, etc.  But here, no recollection at all?  No way to document what led to your strong belief in efficacy of said invention?  Nothing at all?


I feel embarassed for you.

History and Accreditation of ABX Testing/Invention?

Reply #31
Arny, help us out please.  How could your recollection of something this important in your life as to have built an entire persona around it on the Internet be so poor?  I often hear you tell stories about old gear you had, their performance, design, etc.  But here, no recollection at all?  No way to document what led to your strong belief in efficacy of said invention?  Nothing at all?


I feel embarassed for you.

You are very kind.  Hope you have my back the same way as I try to stay on topic and figure out the history of this thing.

Arny has been pointing to the Wikipedia as a proof point of his invention. The Wiki says this:

History[edit]
In 1977 Arnold B. Krueger and Bern Muller, both members of the Southeastern Michigan Woofer and Tweeter Marching Society (SMWTMS), invented the ABX Double Blind Comparator System in order to settle a debate if differences between well constructed and level matched amplifiers are audible, Muller being pro and Krueger being against the possibility.[1]


There is also a similar accounting by Arny's business associate at the time: http://home.provide.net/~djcarlst/abx.htm

The ABX Company designed, manufactured and sold the ABX Double Blind Comparator System, which made possible scientifically valid self run subjective comparisons of audio components. The information on these pages is historical. This web site documents the ABX Comparator System and lists the subjective test results obtained by the company and the Southeastern Michigan Woofer and Tweeter Marching Society, SMWTMS. I was one of six partners who formed and ran the ABX Company from 1980 to 1987.

May 7, 1977 SMWTMS did the first ever audio double blind subjective listening tests. An argument over the audibility of differences between amplifiers at a club meeting in November 1976 resulted in an agreement that a double blind test could settle the question. Just six months later, Arny Krueger gave a lecture on his design of a double blind comparator and the first three double blind tests were done. The results include the first three listed in the Power Amplifier Comparison Table in the data. Thus we credit Arny Krueger and his opponent in the argument, Bern Muller, as the inventors of the ABX Comparator. The agreement to create a company to manufacture comparators was informally made the following summer.


I trust you are now embarrassed for Arny because he never gives credit to his co-inventor, Bern Muller?
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

History and Accreditation of ABX Testing/Invention?

Reply #32
So, I'm glad you're laying this all to rest now.
Thank you for clearing this up, guys. I really appreciate it.
Finally we have some resolution to these very important issues.
Ultimately, the clarity you've provided will be of benefit to us all.

How about we lock this topic now?

History and Accreditation of ABX Testing/Invention?

Reply #33
Indeed, I'm not sure why this thread got onto two pages. It's just men bitching at each other, it's not a discussion and looks decidedly immature and unprofessional. Not that we can't be immature obviously

History and Accreditation of ABX Testing/Invention?

Reply #34
How about we lock this topic now?

I see no reason to do so. The discussion is relatively civil. Participation is optional.

amirm: may I suggest that if it is so important to you to show that Krueger is "merely" a co-inventor of the methodology we today associate with ABX testing (can you at least agree on this statement?) that you then write up a  coherent series of events as you believe they happened and give Krueger a chance to answer coherently if he wishes to correct something or elaborate further on the course of events. This pulling out quote after quote is not conductive to "setting the record straight" if that is the goal here. All you get is more confusion and scattered pieces to the puzzle you are anxious to put together.

The question of "who" invented something, especially if it is an idea and not a physical object, is most often muddled in any good conductive environment. That is certainly true in academia in which I work. Nevertheless the one that does something about it is the one who gets the final credit. You may put acknowledgements in your publication if you think someone who is not a co-author made significant contributions to the definition of the problem or the solution but unless these people spend significant time in helping you with the actual construction of the solution you put in the paper this is merely a courtesy on the part of the author.
But how can you expect to now get an account of who said what doing lunch that prompted who to think of which solution?

History and Accreditation of ABX Testing/Invention?

Reply #35
The “prior art” is quite clear so by formal legal (patent) definition the invention is not one either.
Almost every "invention" has some kind of prior art, and it doesn't necessarily disqualify it from being patentable.

The differences in application (psychoacoustic tests vs audio equipment tests) wouldn't help.
The fact that one is an automated mechanical switcher and the other is a user controlled switch box wouldn't help, because user controlled switch boxes are nothing new.
However, the fact that the newer version allows the user to switch back and forth while retaining the same hidden choice of X is a useful and (unless there is evidence to the contrary) novel difference.

The invention was first realised as a real physical machine that works. The earlier ABX method pre-dates it by 30 years, yet no one else applied it to audio equipment tests. It can't have been particularly obvious to do so.

Also, it's apparent from the write-up that someone appreciated a problem, and invented something that solved it. There was a clear inventive step.

Those who understand patents will have ticked off the four words in bold and concluded that the invention was patentable, in the absence of further prior art to prove otherwise.

Cheers,
David.

History and Accreditation of ABX Testing/Invention?

Reply #36
The “prior art” is quite clear so by formal legal (patent) definition the invention is not one either.
Almost every "invention" has some kind of prior art, and it doesn't necessarily disqualify it from being patentable.

The differences in application (psychoacoustic tests vs audio equipment tests) wouldn't help.
The fact that one is an automated mechanical switcher and the other is a user controlled switch box wouldn't help, because user controlled switch boxes are nothing new.
However, the fact that the newer version allows the user to switch back and forth while retaining the same hidden choice of X is a useful and (unless there is evidence to the contrary) novel difference.

The invention was first realised as a real physical machine that works. The earlier ABX method pre-dates it by 30 years, yet no one else applied it to audio equipment tests. It can't have been particularly obvious to do so.

Also, it's apparent from the write-up that someone appreciated a problem, and invented something that solved it. There was a clear inventive step.

Those who understand patents will have ticked off the four words in bold and concluded that the invention was patentable, in the absence of further prior art to prove otherwise.


All good and relevant points. To shed light on what you have said let me explain the genesis of the ABX Comparator.

It is true that a discussion at an audio club meeting between Bern and I was the kick-off what eventually became the ABX product. About the only thing we agreed on was that the methodology used had to be a DBT.

The starting point for my efforts was a relay box that someone else who was a local builder of pro audio amps had built to demonstrate his gear. It just switched amplifier outputs and it put all of the details of switching including the identity of what was being listened to in plain sight. Its owner was struggling with the widespread perception that all amps sounded different and that in particular there were magic amps that sounded a whole lot better than his.

Another starting point was the fact that I had been building switch boxes for comparing amps and receivers since I was 14 or 15. The most complex one I built (when I was 16 or 17)  switched the inputs and outputs of about 2 dozen pieces of equipment at the hi fi shop I worked for.  I had worked for IBM as both a computer and tabulating machine repair man and tab machines were basically specialized computers based on relay logic, so I was very hip to the techniques for realizing fairly complex machines based on relays.

So starting with a careful inspection of this other simple relay box and conversations with its owner and testing it, I proceeded to build a better relay box.

A listening test committee was formed out of the hifi club and we met weekly at Dave Clark's house to discuss doing amplifier tests. I would bring my most recent effort to these meetings where it was critique. A key point was that all acquisition of parts and circuit design and construction was up to me. Nobody else did any more than look at my relay box of the week and critique it.

Clark's JAES paper describes one stage of the development of my relay boxes:

"
However, another problem quickly became obvious.
There was always an audible relay transition time delay
when switching from A to B. When switching from A
to X, however,the time delay would be missing if
was really A and present if X was really B. This extraneous
cue was removed by inserting a fixed length
dropout time when any change was made. The dropout
time was selected to be 50 ms which produces a slight
consistent click while allowing subjectively instant comparison.
"

What really happened is that once I had devised and implemented  a means for establishing a series of Xs which involved a telephone stepping relay, I immediately noticed this problem. However, the next weekly meeting was the next evening and I had a day job, and no opportunity to deal with this problem. I showed up with the box, pointed out the problem and promised to have it fixed by the next meeting, which I did. The 50 msec was an estimate because I don't think that it was ever measured. I implemented the change over delay feature with a few resistors and capacitors and showed up at the next meeting with the problem solved.

This went on for a number of months as I identified and solved a long list of problems that evolved once each stage of development was solved.  In the end a fully functional relay logic ABX box that served as the only device of its kind for probably a couple of years achieved a stabilized design and just worked every time it was needed.

Bern Muller wanted to have his own ABX box but it was agreed that we were done with relay logic because if its bulk, weight and cost. Bern had achieved a good level of comfort with TTL logic having built his own stereo audio digital delay line. He set forth on his own just like I had before, to develop a TTL ABX Comparitor which was conceived as a driver for external relay boxes for various functions.

Bern's first working model was built using wire wrap techniques. Several of the functions of the original ABX box such as automatically generating a sequence of random X's were not satisfactorily developed at this point.  The TTL wire wrap box design was converted to a PC card design by a thrid and fourth party which went through several generations. I didn't have a lot of hands-on with those phases but eventually I came up with and implemented a solution to the random X problem as an enhancement to a PC-card ABX box, and so on.

A longer term problem was a means for switching the inputs and outputs of integrated amplifiers from phono input to speaker output with no switching transients. I eventually realized that this was all about timing, and I implemented a relay logic box that worked brilliantly and was used for a number years. However, same problem, it was based on the better part of a dozen multi-contact bifurcated gold contact relays that cost an arm and a leg.  I then came  up with a reed relay CMOS logic version that worked just as well. Dave Clark didn't like my timing circuit and replaced with one of his own which worked just as well but had few less chips. That eventually became the ABX RM2 relay module.

So look at this series of anecdotes and who invented what? I assert that I am the sole designer and builder of the first working ABX1982 box. Since that box was the first working implementation of any or all the basic technologies that were unique to it (such as the solution to the timing problem that Clark mentions in his JAES paper) I was sole designer and builder of that box which I used to do the first ABX1982 listening test.

History and Accreditation of ABX Testing/Invention?

Reply #37
The “prior art” is quite clear so by formal legal (patent) definition the invention is not one either.
Almost every "invention" has some kind of prior art, and it doesn't necessarily disqualify it from being patentable.

Hi David.  Thanks for your contributions to here.  As to your point, such prior art would have to clearly be stated in the patent application as to allow the examiner a chance to see if the new application is the same or different. To my knowledge, and I am sure every member here, Arny has never provided the reference to Munson and Gardner paper and their work.  As you know, exclusion of such critical prior art is seen as a serious offense in patent applications and would put the applicant in super bad light should it come to surface later. 

Arny said he knew about such art but has never indicated.

Quote
The differences in application (psychoacoustic tests vs audio equipment tests) wouldn't help.

Well it can if you can demonstrate that the invention works specifically for that domain that makes it unique.

Quote
The fact that one is an automated mechanical switcher and the other is a user controlled switch box wouldn't help, because user controlled switch boxes are nothing new.

Exactly.

Quote
However, the fact that the newer version allows the user to switch back and forth while retaining the same hidden choice of X is a useful and (unless there is evidence to the contrary) novel difference.

We only have the abstract of Gardner and Munson paper.  It is possible it too had thought about this variation.

But let's say that is not the case and Arny was the one to have thought about this variation.  Then the invention is what you described: user control of an ABX test.  Not the ABX test itself.  That would have been covered by the prior art and not an invention.

Quote
The invention was first realised as a real physical machine that works. The earlier ABX method pre-dates it by 30 years, yet no one else applied it to audio equipment tests. It can't have been particularly obvious to do so.

Taking that argument as is, the current ABX tests using computer programs does not do that.  It is indeed far closer to the original 1950 ABX system than any physical box built.  In that sense then, Arny should never mention his "invention" in the modern context where we are talking about software comparators.

Quote
Also, it's apparent from the write-up that someone appreciated a problem, and invented something that solved it. There was a clear inventive step.

Design engineers appreciate problems all the time and build solutions to it.  That doesn't make them inventions.  You yourself said correctly that switching audio equipment is nothing new.  So that aspect by itself is not patentable.  So we can't use that, i.e. building a switch box, to now justify said invention claim.

Quote
Those who understand patents will have ticked off the four words in bold and concluded that the invention was patentable, in the absence of further prior art to prove otherwise.

That seems like a shot across my bow .  See this for a small indication of my knowledge there: http://www.boliven.com/patents/search?q=(%...+Majidimehr%22).  I had to deal routinely with patent claims when I was at Microsoft.  While no one other than a specialized patent attorney can give proper evaluation of such things, I am no stranger to the topic.  Or else, I would not have created this thread.  Based what I know, Arny has built a swtichbox, not invented ABX.

What you are missing in your list is one of the key aspects of patent law in many countries: test of non-obviousness.  See: http://en.wikipedia.org/wiki/Inventive_ste...non-obviousness

The inventive step and non-obviousness reflect a same general patentability requirement present in most patent laws, according to which an invention should be sufficiently inventive — i.e., non-obvious — in order to be patented.[1] In other words, "[the] nonobviousness principle asks whether the invention is an adequate distance beyond or above the state of the art."[2]

Had this been a true patent application and Arny had correctly referenced prior art in the form of the 1950 disclosure, the application would have been rejected with speed of light.  Why?  How could it be more obvious?  The original test names the scheme ABX.  What does Arny name it?  ABX!  Clearly that name came about because what he built was a box implementing said scheme.  It is obvious to anyone schooled in the art, i.e. a design engineer, that building an implementation of a prior invention is not an invention by itself. 

You could patent the novel aspects of the circuit design if there were any.  But not the functionality of the box.  The box implements what the original art says: presents the user with A, B and asks if X is if X is closer to A or B.  If I polled a 100 people here without the context of this thread, do you think they would say something different about ABX test methodology?  They would not, right?  So the "invention" we use and care about now is what was disclosed in 1950.  None of us use a hardware ABX comparator so saying one "invented" that is not material, nor does it provide a distinction of value to members discussing matters related to blind testing.

Thanks again.
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

History and Accreditation of ABX Testing/Invention?

Reply #38
I assert that I am the sole designer and builder of the first working ABX1982 box. Since that box was the first working implementation of any or all the basic technologies that were unique to it (such as the solution to the timing problem that Clark mentions in his JAES paper) I was sole designer and builder of that box which I used to do the first ABX1982 listening test.

So say it that way Arny.  You would get no argument from me.  Instead, you say stuff like this: http://www.avsforum.com/forum/91-audio-the...ml#post21669957

[quote author=arny on AVS link=msg=0 date=]For me, its about working to the highest possible standard. In the case of ABX, I invented the standard.[/quote]

You see how different your two statements are? You didn't invent the "ABX standard."  The credit for that goes to Munson and Gardener.  That was the standard. You are the designer and builder of a switchbox implementing that invention.
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

History and Accreditation of ABX Testing/Invention?

Reply #39
Take it up over there. This topic will now close.

I've opened the discussion to allow further discussion.  When the quotes have all come from another forum, in which the two main parties are actively involved, it seems strange to air the laundry over here.

Nevertheless...
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?

History and Accreditation of ABX Testing/Invention?

Reply #40
Take it up over there. This topic will now close.

I've opened the discussion to allow further discussion.  When the quotes have all come from another forum, in which the two main parties are actively involved, it seems strange to air the laundry over here.

Nevertheless...

That thread was unlocked when I started typing a response, and it was locked while I was editing the response a few minutes later. Therefore it was lost.

History and Accreditation of ABX Testing/Invention?

Reply #41
Thank you so much for opening the thread.  The original post that started this thread was indeed from this forum.  But just to address the criticism anyway, here are further references in this forum:

I dumped vinyl in 1984 which was about 7 years after 1977 when I invented ABX.


Not weird - totally expected. Perceptions exactly like those stated above is the big reason why I invented ABX.


The above paragraph pretty much describes our state of mind when we invented ABX.

Notice the switch from "I" to "we."

I invented ABX to unmire myself from the false thinking and claims of high end publications like the Atkinson Stereophile and TAS.

Back to "I" .

This from a locked thread:  http://www.hydrogenaud.io/forums/index.php...st&p=697064
Original post from Epstein: About the only thing you've said is your claim to have "invented" ABX, which I take to mean your particular variant of standard double blind AB testing, which is something that I was involved in with a group of associates in the early to mid '70s.

Arny's confident answer: There is a JAES paper that dates and validates my claim.  Got game?

Based on information in this thread we now that Arny did not have game either .

Here is why we need this thread:

[And thanks for inventing ABX, by the way. That was a huge contribution, unlike this relatively trivial matter!]


Clearly mzil thinks Arny invented "ABX."  Not some hardware switch box that was built in 1980s but giving him credit for what was invented 60 years ago.

The search goes on and on but I will stop here .
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

History and Accreditation of ABX Testing/Invention?

Reply #42
The “prior art” is quite clear so by formal legal (patent) definition the invention is not one either.
Almost every "invention" has some kind of prior art, and it doesn't necessarily disqualify it from being patentable.

Hi David.  Thanks for your contributions to here.  As to your point, such prior art would have to clearly be stated in the patent application as to allow the examiner a chance to see if the new application is the same or different. To my knowledge, and I am sure every member here, Arny has never provided the reference to Munson and Gardner paper and their work.  As you know, exclusion of such critical prior art is seen as a serious offense in patent applications and would put the applicant in super bad light should it come to surface later.
But Arny didn't file a patent application, so it's unfair to imply that he has done anything wrong here.

(I think this requirement to disclose known prior art alongside patent applications is a North American thing btw. I don't think it's universal.)


Quote
Quote
The invention was first realised as a real physical machine that works. The earlier ABX method pre-dates it by 30 years, yet no one else applied it to audio equipment tests. It can't have been particularly obvious to do so.

Taking that argument as is, the current ABX tests using computer programs does not do that.  It is indeed far closer to the original 1950 ABX system than any physical box built.  In that sense then, Arny should never mention his "invention" in the modern context where we are talking about software comparators.
That impacts the patentability in some jurisdictions. The USA seems happy enough to accept patents (or at least, claims in patents) for machines running computer code, where it's only the content of that computer code that's inventive.


Quote
Quote
Those who understand patents will have ticked off the four words in bold and concluded that the invention was patentable, in the absence of further prior art to prove otherwise.

That seems like a shot across my bow .
No, it wasn't meant as such.

Quote
See this for a small indication of my knowledge there: http://www.boliven.com/patents/search?q=(%...+Majidimehr%22).  I had to deal routinely with patent claims when I was at Microsoft.  While no one other than a specialized patent attorney can give proper evaluation of such things, I am no stranger to the topic.  Or else, I would not have created this thread.  Based what I know, Arny has built a swtichbox, not invented ABX.
He seems to have invented a form of ABX, if not the initial concept. He seems to have had the idea of applying it to audio equipment.


Quote
What you are missing in your list is one of the key aspects of patent law in many countries: test of non-obviousness.  See: http://en.wikipedia.org/wiki/Inventive_ste...non-obviousness

The inventive step and non-obviousness reflect a same general patentability requirement present in most patent laws, according to which an invention should be sufficiently inventive — i.e., non-obvious — in order to be patented.[1] In other words, "[the] nonobviousness principle asks whether the invention is an adequate distance beyond or above the state of the art."[2]

Had this been a true patent application and Arny had correctly referenced prior art in the form of the 1950 disclosure, the application would have been rejected with speed of light.  Why?  How could it be more obvious?  The original test names the scheme ABX.  What does Arny name it?  ABX!  Clearly that name came about because what he built was a box implementing said scheme.  It is obvious to anyone schooled in the art, i.e. a design engineer, that building an implementation of a prior invention is not an invention by itself. 

You could patent the novel aspects of the circuit design if there were any.  But not the functionality of the box.  The box implements what the original art says: presents the user with A, B and asks if X is if X is closer to A or B.  If I polled a 100 people here without the context of this thread, do you think they would say something different about ABX test methodology?  They would not, right?  So the "invention" we use and care about now is what was disclosed in 1950.  None of us use a hardware ABX comparator so saying one "invented" that is not material, nor does it provide a distinction of value to members discussing matters related to blind testing.
I did mention "obvious" - it was one of the words I put in bold.

So there's a kind of test that exists (had existed for nearly 30 years at the time). No one has applied it to this different kind of testing. No one has allowed the user of the test to interactively control the switching or the timing of the test. No one has designed the circuits to allow the user to take control in this way. No one has designed the circuits to allow the user to repeat a single trial of the test until they are happy with it. Then one person comes along, does all this, but apparently it's obvious, not inventive, and they couldn't have got a patent?

You may or may not have read more patents than me, but I bet you've seen granted US patents that are far more obvious, and far less inventive.

Cheers,
David.

History and Accreditation of ABX Testing/Invention?

Reply #43
The box implements what the original art says: presents the user with A, B and asks if X is if X is closer to A or B.


The above is a false claim, as reference to the journal abstract that Amir himself cited shows:

http://scitation.aip.org/content/asa/journ....1121/1.1917190

"An observer is presented with a time sequence of three signals for each judgment he is asked to make. During the first time interval he hears signal A, during the second, signal B, and finally signal X. His task is to indicate whether the sound heard during the X interval was more like that during the A interval or more like that during the B interval. "

Note that Amir's reference mentions three time periods while Amir's paraphrased version of it dropped one of the periods presumably in the interest of making the procedure in the 1950 paper more closely resemble the ABX procedure in Clark's 1982 paper. However, the 1982 paper adds a number of features and operations and also provides a detailed schematic description of an electronic circuit that implements them. The 1982 schematic is 100% solid state and included integrated circuits which did not exist in 1950 so there is no chance the two papers describing similar implementations.

The 1982 ABX Comparator's function is as described in excerpts from Clarks 1982 JAES paper previously placed on this thread. Amir has quoted from this paper so he is obviously as familiar with it as he ever seems to get.  I will summarize it as follows:

An observer is allowed to freely select any of three signals, A, B, and X which is secretly and automatically chosen from A and B, until he can indicate whether the sound heard during the X intervals was more like that during the A intervals or more like that during the B intervals. The listener's choices are safely and secretly stored until the test is over. When he ends the test the listener is informed of the correctness of his choices. During the listening periods the timing of the sounds is slightly modified so that the identity of the X interval can only be determined by means of its audible properties and not any switching artifacts.

http://www.legalzoom.com/patents-guide/wha...patentable.html

A patent cannot protect an idea. Instead, the idea must be embodied in one or more of the following:
A process or method (such as a new way to manufacture concrete)
A machine (something with moving parts or circuitry)
A manufactured article (such as a tool or another object that accomplishes a result with few or no moving parts, such as a pencil)
A new composition (such as a new pharmaceutical)
An asexually reproduced and new variety of plant.

The description of ABX1982 above and the Clark 1982 JAES paper describe both a new process or method and a new machine that differ from previous art in non-obvious ways.

History and Accreditation of ABX Testing/Invention?

Reply #44
I have to point out that the "abx" test of Munson was a sequential test, not a time-proximate test in which the subject had control over stimulus selection at all points in time.  This test has been brought up from time to time, and various people have taken upon themselves to scold the entire AES about how this sequential test is such a bad test (as it is, the lack of time proximity is massively desensitizing), ignoring the fact that this is NOT the modern test called "ABX".

The ABX test from the Michigan bunch was time-proximate.  This is a substantial improvement, and is what is commonly referred to in the present day as the ABX test.  There may still be an issue of silences inserted, or clicks. It's a difficult problem to solve with 1960's technology, but it's at least mostly time-proximate.

Modern ABX tests using windowed digital switching are even better, of course.

Much of an ABX test, however, comes in the material selection, listener training, listener comfort, and presence of listener feedback.
-----
J. D. (jj) Johnston

History and Accreditation of ABX Testing/Invention?

Reply #45
I have to point out that the "abx" test of Munson was a sequential test, not a time-proximate test in which the subject had control over stimulus selection at all points in time.  This test has been brought up from time to time, and various people have taken upon themselves to scold the entire AES about how this sequential test is such a bad test (as it is, the lack of time proximity is massively desensitizing), ignoring the fact that this is NOT the modern test called "ABX".

The ABX test from the Michigan bunch was time-proximate.  This is a substantial improvement, and is what is commonly referred to in the present day as the ABX test.  There may still be an issue of silences inserted, or clicks. It's a difficult problem to solve with 1960's technology, but it's at least mostly time-proximate.

Modern ABX tests using windowed digital switching are even better, of course.

Much of an ABX test, however, comes in the material selection, listener training, listener comfort, and presence of listener feedback.


Totally agreed about the effectiveness of any proper test being heavily dependent on "...listener training, listener comfort, and presence of listener feedback". We noticed this almost immediately after we did our first DBTs. 

There are two divergent paths that start at this point. One is the selection and development of artificial sounds such pink noise that are more diagnostic for certain technical flaws such as frequency response variations. The other approach is sorting and selecting from actual recordings, either purpose built (e.g. SQAM) or what amounts to be random selections from contemporary or archival recordings.  Actual commercial recordings are probably the source whose relevance is most clear to the general public and audiophilia, but the process of finding them is almost completely random.

To me one of the clear indications of the falseness of uncontrolled audiophile listening tests is their absence of concern with effective listener training, selection of program material and specific operational circumstances. IME listener training has many of the elements of training for an athletic event wherein an actual assessment of listener performance quality (correctness) is quickly fed back to the trainee for his benefit.

Obviously, a listening methodology with a high propensity for false positives such as sighted evaluations has very limited benefit to the process of listener training.  No pain, no gain.

History and Accreditation of ABX Testing/Invention?

Reply #46
I have to point out that the "abx" test of Munson was a sequential test, not a time-proximate test in which the subject had control over stimulus selection at all points in time.

Hi JJ.  Thanks for chiming in.  What you describe was very clear in the original abstract from ASA.  I think giving control to the user a variation of such a test.  It is not an all new invention in my opinion.

Quote
This test has been brought up from time to time, and various people have taken upon themselves to scold the entire AES about how this sequential test is such a bad test (as it is, the lack of time proximity is massively desensitizing), ignoring the fact that this is NOT the modern test called "ABX".

The ABX test from the Michigan bunch was time-proximate.  This is a substantial improvement, and is what is commonly referred to in the present day as the ABX test.  There may still be an issue of silences inserted, or clicks. It's a difficult problem to solve with 1960's technology, but it's at least mostly time-proximate.

If there is no source control for segment selection and looping, I don't think adding user control gets us to where we want to be.  Yes, it is better than timed switching.  But one can't compare the past to the future as was done with the ABX comparator itself. 

Quote
Modern ABX tests using windowed digital switching are even better, of course.

They certainly are a huge improvement over the real-time versions with just the output switching.  I think you agree that the less we are "time proximate" the more the outcome tilts toward negative outcome.  User gets frustrated that he can't recall the the other track and will vote randomly or "always A." 

Quote
Much of an ABX test, however, comes in the material selection, listener training, listener comfort, and presence of listener feedback.

A failing unfortunately for just about every test cited on audio forums as proof of this and that.  The outcomes may still be right but the protocol certainly not.

I would also add to your list poor user interface.  I find foobar ABX plug-in woefully inadequate for finding small differences.  Its segment selection interface is horrid on full track for example.  Lack of recall for the segments once you find them makes it worse.  All of this again tilts the odds toward the "house" by generating negative outcomes.

The flaws are not material if we are talking about gross distortions that are audible to masses.  But the moment we start to say that it applies to any and all people, I think we are being improper and unfair.

So in that sense, the proper ABX test for finding small differences is yet to be invented.  In MPEG as you know we have our list of codec killers.  Where is that list for jitter over HDMI?  Where is that list for differences between amplifiers?  The people who obsess over double blind ABX tests are seemingly very comfortable with negative outcomes so there is no desire to find the scenarios that are most revealing.  The people who believe there are differences don't care about ABX testing, nor have the necessary knowledge.

Net, net, when these tests are applied in forums and among hobbyists, I believe it is a biased game aimed to produce one outcome more than the other. 

Culturally we need to make it OK and "safe" in corporate speak to criticize the tests and their outcomes.  Only then can we create the much more sensitive and defensible results we desire when differences get small.
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

History and Accreditation of ABX Testing/Invention?

Reply #47
I find foobar ABX plug-in woefully inadequate for finding small differences.

What small differences??
What alternate method are you using to determine the existence of these audible "small differences", that foobar ABX et al are "woefully" inadequate for finding?
MUSHRA? Or...??

Quote
Much of an ABX test, however, comes in the material selection, listener training, listener comfort, and presence of listener feedback.

A failing unfortunately for just about every test cited on audio forums as proof of this and that.  The outcomes may still be right but the protocol certainly not.

Amir, instead of sniping from the bushes, how about eliminating the guesswork about these outcomes and run a robust BS-1116 of any of these negative nelly nulls, produce some positives, so that you demonstrate that indeed, these tests are missing those audible "small differences", that the GE's "hear" with ease under completely uncontrolled conditions.
That's what drives these blind tests to begin with, remember?

cheers,

AJ


Loudspeaker manufacturer

History and Accreditation of ABX Testing/Invention?

Reply #48
I would also add to your list poor user interface.  I find foobar ABX plug-in woefully inadequate for finding small differences.  Its segment selection interface is horrid on full track for example.  Lack of recall for the segments once you find them makes it worse.  All of this again tilts the odds toward the "house" by generating negative outcomes.

The foobar ABX facility is much more convenient to navigate with shorter samples of music lasting just a few minutes.

As regards recalling of segments when using the foobar ABX plug-in, I simply use a pen and paper to note down the start and finish time of a segment of interest.

I have to wonder if such minor obstacles are enough to preclude hearing an effect, whether the effect is material for the listener's hearing.

Quote
In MPEG as you know we have our list of codec killers.  Where is that list for jitter over HDMI?

Is jitter over HDMI audible even with artificial tones? 

Perhaps the lack of killer samples for jitter reflects the lack of HDMI jitter being audible in practice for human listeners.  (I seem to recall Ethan Winer suggesting something to the effect that jitter should not normally be noticeable with modern equipment.)

It is one thing to use a spectrum analyser to show the existence of jitter components many db down at frequencies adjacent to a test frequency. It is another for our brains to detect such components, based on nervous impulses from the organ of Corti. We are but flesh and blood!

I have never heard an audio file that incorporates an intentionally large amount of jitter, introduced for demonstration purposes, to hear what really excessive jitter actually sounds like.  (Perhaps someone here at HA might be kind enough to provide a link to such a file.)  Decades ago, I heard tape wow and flutter when using a domestic open reel tape recorder. I can only imagine how untidy the spectrum of a test tone might look like when played back on even a professional open reel recorder operating at 15 ips. However whether the frequency wobbles would be audible, is another matter.

History and Accreditation of ABX Testing/Invention?

Reply #49
There are two divergent paths that start at this point. One is the selection and development of artificial sounds such pink noise that are more diagnostic for certain technical flaws such as frequency response variations. The other approach is sorting and selecting from actual recordings, either purpose built (e.g. SQAM) or what amounts to be random selections from contemporary or archival recordings.  Actual commercial recordings are probably the source whose relevance is most clear to the general public and audiophilia, but the process of finding them is almost completely random.



This, to me , is a key point. 

Researchers can set up extremely sensitive comparison protocols to detect extremely small differences...as JJ has noted, down to the the absolute physiological limit of audible level difference.

If difference is detected that way, though, it is an overreach to claim (as audiophiles are inclined to do) that this 'proves' the  differences *they* 'heard', sighted, were real.

IOW, just because we can demonstrate a difference in a lab setting with pink noise or pure tones or at unusually elevated levels or with specially selected gear or after careful training to hear artifacts, doesn't mean it's something people heard when listening normally to music.  Even 'golden ears'.  The 'most revealing' protocol may have academic or technical value but little or no relevance to real world  listening.

NB both Meyer and Moran (2007) and the Meridian group (2014) used music as their difference probe.

 
SimplePortal 1.0.0 RC1 © 2008-2019