Quote from: John Eppstein on 2010-03-28 05:01:11

No. Not when we're evaluating recording equipment and what we're looking at is the cumulative effect that the recording chain and process has on the resulting recording. The recording is not the source, it is the product.

No. In an ABX test of recordings you would compare recording A with recording B. If a listener cannot find a difference between A and B the recordings would appear to be transparent with regard to each other for that listener.

You are incapable of arguing anything without using logical fallacies it seems (I've seen at least four now), so I'm done with this. Keep living in your fairytale world.

There are circumstances where no good scientific test is possible, but that's a defect, not a recommendation, particularly when the differences get subtle and bias and confabulation become far more likely.

p.s.  I can't resist adding this beautiful example by Feynman of the power of bias.  The only thing I would add is that it is now and will ALWAYS be a problem, even in the sciences, because sometimes there are very powerful desires behind a bias, and scientists are by no means immune:

Quote

We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It's a little bit off because he had the incorrect value for the viscosity of air. It's interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher. Why didn't they discover the new number was higher right away? It's a thing that scientists are ashamed of--this history--because it's apparent that people did things like this: When they got a number that was too high above Millikan's, they thought something must be wrong--and they would look for and find a reason why something might be wrong. When they got a number close to Millikan's value they didn't look so hard. And so they eliminated the numbers that were too far off, and did other things like that. We've learned those tricks nowadays, and now we don't have that kind of a disease.

Quote from: John Eppstein on 2010-03-28 05:01:11

It's not that ABX testing, used correctly, is invalid. The key phrase is "used correctly". Some people appear to believe that ABX is some kind of panacea - that if it can't be proved by ABX it doesn't exist and that is simply not true.

Point us to one.

If I may address all, please, folks, realize that the creative side and the technical side are both important in this industry, and it woudl be remarkably nice, not to say unusual and amazing, if we could get along with each other once in a while.

John, ABX testing really does work for determining if a given subject can hear a difference, when it's done right.  Nobody can prove a negative, not a scientist or anyone else, but there is valid statistical inference regarding populations from repeated test results, being negative or positive. There is never any absolute, but a probability suggesting that the age of the universe is involved is certainly definitive.

Techie guys, you know, the artistic side tries to lay down what sounds good. Euphony is their stock and trade. There's more than a little bit to be gained in the knowledge they have, as well, rather than simply telling them that their tape deck distorts and isn't as accurate. It's NOT as accurate, but it might just sound better, since rather obviously, what we capture in the studio or anywhere else is about, oh, what, 1/10000th of the analytic information in the soundfield around the listener's head.


So, how about some friendly dialog?

Hydrogen audio may not be the best place, I suppose, you know, you CAN accurately measure a given person's preference, but with an ABC/hr test, so strictly speaking, it wouldn't be ABX, it would be another test.  So, I suppose purely testing preference would violate TOS 8.

But given the ranting I get from audiophiles (check out the talk announcement I put up on stereophile board), I understand why TOS 8 exists. Some arguments just aren't worth having.

So, dunno an answer, but fighting accomplishes nothing.

4) Now about perceptual state of the brain, while fascinating, I'm not sure it invalidates the abx test. But maybe it can be useful to study how a trained human being can perceive things that others don't.

malice

Quote from: krabapple on 2010-03-28 08:45:01

Point us to one.

If you can't see it, you can't see it. Sorry.

What bugs me are people who assume that we have everything we need to understand everything (which is patently absurd).

What also bugs me are people who attempt to misuse a tool like ABX to try to "prove" a negative, which is also patently absurd.

So John is what you're saying is that the same basic kinds of performance and orthodox electronic theory that make a digital convertor good for playback does not make it good for recording?

Joh, you are picking the wrong pliers. Try the pliers that orthodontists use to adjust braces. Your mistake here is that you are comparing a brute force tool to a fine adjustment tool, which invlidiates your comparison of dentisty to jewelry making.

Quote from: malice on 2010-03-28 16:57:39

Quote from: Arnold B. Krueger on 2010-03-28 13:46:38

Recordists are invulnerable to bias?

Not so much! ;-)

I don't believe recordists are invulnerable to bias (as I don't share many of John's view, mind you)

I do believe that recordists (and I mean professional experienced recordists) have one major advantage as matter as bias is concerned. Good engineer mixers (especially mixers) know that bias is the number one concern in their field. Mixers do know that the mind is a very tricky thing that will make their mixes sound good while they are "not so good yet". Same applies to ME and  I think it's Bob Katz that said once : "you will only have one chance to hear a mix for the first time". I would be very curious to learn more about a serious research about this topic of bias and how it affect our judgement and how we integrate the fact that we KNOW we are vumlnerable.

It turns out that Katz and Massenberg have granted crediblity to blind listening tests in public statements. I know of no hand-waving from them about fear and loathing and performance anxiety.

Quote

Quote

Your first problem John is that Ethan and I are recordists. I know that Ethan records well because I've heard his work. The recordings I make are copyrighed by a number of someone else's so I can't freely disseminate my work like Ethan does. And, they are not made using professional artists in professional grade venues. So I don't have that to wave around. I can fearlessly predict that other than an outright miracle of God, I'll never be credited with anything to do with an award-winning recording! However, I probably have in the past 10 years made more recordings by accident than most professional recordists have made on purpose.

I do not know about your credentials in matter of recordings, and I certainly won't evoke what I know about Ethan's work.

I don't know about yours Malice. I asked you to provide your credentials privately several days back and you are a no-show.

As far as Ethan's work goes, it is a jillion times better than what is available to me of yours, because something  that is even just halfways-decent always beats absolutely nothing.

Quote

You did notice I was not good enough in the Electrical engineering side.

To say the very least!  This raises further issues about crediblity credibility because you did give your statements as if you thought you were some kind of authority in the matter. "I don't know" is a fair answer that gets respect. So is silence.

Quote

I might be wrong, and I hope you'll forgive me about this, but I've seen comments from you that tells me you would be able to handle a very long argument about mixing with me or mixerman.

That  statement is  trash talk because you have provided no credentials, despite you making an issue of the matter and failing to provide them, even privately.  The womb has proven itself to be far from being a paragon of free speech.

Quote from: Arnold B. Krueger on 2010-03-28 13:46:38

So John is what you're saying is that the same basic kinds of performance and orthodox electronic theory that make a digital convertor good for playback does not make it good for recording?

What I'm saying is that a higher degree of precision is desirable for recording equipment; consumer level digital gear imparts characteristics that are, for lack of a better term, uneuphonious. Actually, ALL pcm digital equipment has some degree of undesirable sonic character, but it is more pronounced in the lower quality stuff, yes. That's not to say that just because something is expensive it's automatically good, of course.

I don't believe recordists are invulnerable to bias (as I don't share many of John's view, mind you)

Hey Arnold, I've repeatedly asked you for credentials and you haven't shown me squat.

Checking online all I find is a raft of forum posts. I've told you more about my experience than you have told me about yours.

About the only thing you've said is your claim to have "invented" ABX, which I take to mean your particular variant of standard double blind AB testing, which is something that I was involved in with a group of associates in the early to mid '70s.

Oh, and that you favor cheap junk gear over quality gear.

Maybe I've missed the point, but I don't see why the discussion has to be rancorous.  It's certainly possible that abx testing or the other tests that are well-accepted here do not cover all forms of listening perception or even interfere with those other forms of perception.  But what is the way of proving the existence of that other form of perception?  What is the alternative testing to be used? 

It's as if one were identifying a forgery or a wine knockoff from the originals.  Someone may do it through meticulous concentration on brushstrokes or acidity.  Someone else react to the whole, and say the painting or wine lacks depth, warmth, richness etc.  Fine.  There's no problem either way, whatever perceptive capacity they exercise, if they can consistently identify the painting or wine without cheating. 

All the tests used in this forum are designed to rule out guessing and cheating (and the worst kind is the unknowing or unintentional one of bias).  I don't see any particular theory required by any of the tests, except for commonplace observations about the prevalence of bias, mechanisms to avoid or neutralize that bias, and the use of well-established laws of statistics.   

So why any rancor?  There's no attachment to a theory or not much of one, and no attachment even to a particular test, if there's another test that can rule out bias and guessing.

Quote from: malice on 2010-03-29 11:12:26

Incidently, I wonder why I cannot find an hi-fi magazine or a pro-audio magazine totally unbiased because of  their sponsors.

There's at least one on-line hi-fi "magazine" which is subscription only, and carries no adverts. It claims to be unbiassed. (If I could remember what it was called, I'd say - I'm being forgetful, not coy!)
Cheers,
David.

Quote from: TechVsLife on 2010-03-28 19:57:25

It's certainly possible that abx testing or the other tests that are well-accepted here do not cover all forms of listening perception or even interfere with those other forms of perception.  But what is the way of proving the existence of that other form of perception?  What is the alternative testing to be used?

Indeed. It is possible that ABX tests do not cover all forms of listening perception. Russell's Teapot is worth considering here, even if it was initially proposed in a different context:

Quote

If I were to suggest that between the Earth and Mars there is a china teapot revolving about the sun in an elliptical orbit, nobody would be able to disprove my assertion provided I were careful to add that the teapot is too small to be revealed even by our most powerful telescopes. But if I were to go on to say that, since my assertion cannot be disproved, it is an intolerable presumption on the part of human reason to doubt it, I should rightly be thought to be talking nonsense.

If not ABX then what? What would these tests look like? ABX-detractors do not go so far as to identify alternate procedures that contain adequate safeguards against the critical errors caused through placebo and sighted-testing. They do not do so for one simple reason: They lack the nuanced understanding of statistics and science required to create such a novel methodology.

Double-blind tests are one of the gold standards of scientific testing. Where they cannot be undertaken, researchers frequently complain that they wish they could be.

Quote from: TechVsLife on 2010-03-28 19:57:25

...But what is the way of proving the existence of that other form of perception?  What is the alternative testing to be used?...

You can't prove if it doesn't exists.
I don't know why some people here don't understand that brain hears what comes through the ears and than imagine or fabricates extra information according to ones biases and mental health. The ABX test eliminates that extra imagined "information". There are people out there that hear voices that nobody else can (schizophrenia). Does that mean they have a better hearing?!

It doesn't matter how euphonic your imagination is. It is irrelevant to anybody else because it is inside your head only. It doesn't exist. That's why it would not survive a double blind test.

There is no such a thing as somebody that have a better discernment in a test that is not double blinded. The double blinded test gives you always a better discernment. Otherwise your "taste" and inclination to like more certain brands over others, added to, lets say, low confidence level, desire to blend in and other psychological deformations will drive you to choose what is more complex and more expensive instead of what actually makes sense.

The second argument is about the euphonic qualities that we are debating for several days.
It refers to how some people, like myself, feel that small amount of distortion can increase  apparent depth and dimension, and therefore, sounds "better".

And that would be an impossible can of worms to debate here because it's impossible to know if that "better perception" is caused by the increase of the harmonic distortion components that increase the overall level or by a real "improvement" in depht and dimension caused by the said distortion.

So yes: very difficult to prove or disprove this but that is definitely worth considering when you compare a good tape recorder and a converter or an analog EQ with big Iron transformers in it, and a plug-in EQ.

malice

Quote from: malice on 2010-03-29 11:07:49

Ok, I did make some research about that 0.1 db increase, and I'm afraid I cannot yet post that post from Bob Katz as it is part of a private forum that forbids me to quote exactly or refeer about the sources.

People can say whatever they want. I can say that I've heard 0.01 dB level differences but oh darn, I've lost track of the actual details. ;-)

Back in the 80s there was a guy named Peter Moncrieff who claimed to hear differences measured in millibels. Strangley enough nobody was ever able to duplicate his work in a bias controlled listening test. ;-)

Quote

But I can elaborate on the very interesting things that has been said and that shows how difficult it is to evaluate anything analog in tests like ABX. Not saying impossible, but you have to take in account what you are testing.

The first argument was about the drift of some analog devices shows, even with 20-turn trim posts, calibration can change, and 0.1 dB is not out of question. If you add a very slight of load, you can end up with another 0.1 dB and you're off the charts of proper abx testing.

I've been level-matching all sorts of audio equipment, even relatively unstable equipment such as analog tape and vinyl playback, since the late 1970s. Analog tape and vinyl can definately drift or wobble during playback by up to 1  dB. Normal signal-handling equipment such as amplifeirs, preamps, equalizers and the like have no problem staying within 0.1 dB.

Very shortly after we started doing ABX, I developed a precision level-matching device that was composed of  a pair of 10-turn potentiometers. I put crank handles on the pot shafts, and it was called "The Fishing Reels". It was basically overkill because we ended up adjusting it with big moves. I've had no problems setting levels within 0.1 dB using  RN50 single turn linear taper potentiometers.

Quote

The second argument is about the euphonic qualities that we are debating for several days.
It refeers to how some people, like myself, feel that small amount of distortion can increase  apparent depht and dimention, and therefore, sounds "better".

As soon as there is an actual audible difference, then how it perceived needs to be evaluated on a individual basis. In some cases ditortion can be added to compensate for an undesirable distortion that crept in from someplace else. What a given kind of distortion sounds like is usually very dependent on the actual music you listen to through it.

Quote

And that would be an impossible can of worms to debate here because it's impossible to know if that "better perception" is caused by the increase of the harmonic distoprtion components that increase the overal level or by a real "improvement" in depht and dimension caused by the said distortion.

Here we see yet another example of an unfortunate choice of words that may lead to false beliefs.

There is no such thing as harmonic distortion as a property of equipment.

Harmonic distortion is an artificial theoretical concept that is used to characterize or measure nonlinear distortion.

All nonlinear distortion creates both harmonics and intermodulation products. However, low pass filtering can prevent harmonics from ever being measured. 

Nobody can ever increase harmonic distortion without also increasing intermodulation distortion. In fact it is possible for equipment to be nonlinear and produce no measurable harmonic distortion, but it will still produce measurable IM. 

For example,  back in the days when resistive networks were used to build DACs there could be signficiant nonlineariities in the resistive network but the reconstruction filter would strip off the harmonics for test signals above about 6.5 KHz. This would create an apparent exception to the usual rule of nonlinearity in electronics increasing at the highest frrequencies. 

Intermodulation distortion produces non-harmonic or aharmonic distoriton as it acts on real-world music, which is more likely as being perceived as being undesirable because it is not musical.

Quote from: John Eppstein on 2010-03-29 13:43:38

Hey Arnold, I've repeatedly asked you for credentials and you haven't shown me squat.

Back at you, John.

Quote

Checking online all I find is a raft of forum posts. I've told you more about my experience than you have told me about yours.

I don't think that's true at all.  I don't think I mentioned that I have a BS in Engineering, for whatever that is worth this far down the road.

Quote

About the only thing you've said is your claim to have "invented" ABX, which I take to mean your particular variant of standard double blind AB testing, which is something that I was involved in with a group of associates in the early to mid '70s.

There is a JAES paper that dates and validates my claim. 

Got game?

Quote

Oh, and that you favor cheap junk gear over quality gear.

So John, in your book Fluke, HP and Audio Precison is cheap junk?

Interesting.

is it because the distortion is in effect, replacing some component of the sound that was lost earlier in the recording process, say when the mic converts the 3D sound field to an electrical signal?

I think the show me your weiner or else I won't accept your authority posts could be removed from the thread. Actual arguments and references to data instead of credentials would make much more sense for the rest of us.

But seriously, I don't think I've ever said that properly applied double blind testing is not valid. Nor have I stated unequivocally that the phenomena we're discussing are untestable by ABX or other double blind tests. All I've said is that the absence of a result in any given test group does not constitute proof that the phenomena don't exist. There are, as you noted, some things that just can't be tested in this way, which is unfortunate.