Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: ABX testing vs. perceptual states (Read 23944 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

ABX testing vs. perceptual states

Reply #25 –
Quote from: John Eppstein on
No. Not when we're evaluating recording equipment and what we're looking at is the cumulative effect that the recording chain and process has on the resulting recording. The recording is not the source, it is the product.

No. In an ABX test of recordings you would compare recording A with recording B. If a listener cannot find a difference between A and B the recordings would appear to be transparent with regard to each other for that listener.

You are incapable of arguing anything without using logical fallacies it seems (I've seen at least four now), so I'm done with this. Keep living in your fairytale world.
"We cannot win against obsession. They care, we don't. They win."

ABX testing vs. perceptual states

Reply #26 –
Quote from: Light-Fire on
Quote from: John Eppstein on
...In the recent issues of Scientific American...

Not a scientific publication. It is not to be taken seriously. It is as trustworthy as any celebrity tabloid.



I haven't read it in years, so I don't know if it's fallen that far.  But back in the day it was only as good as its authors and editors, since there was no peer review. As a result, you still got lots of good science but also the occasional dubious article from a scientist whose work was either faddish and did not hold up, or not respected in his/her field even at the time of publication.

ABX testing vs. perceptual states

Reply #27 –
Quote from: John Eppstein on
Quote from: TechVsLife on
The "source" is not the performance. The source is the recording.

No. Not when we're evaluating recording equipment and what we're looking at is the cumulative effect that the recording chain and process has on the resulting recording. The recording is not the source, it is the product.

Well sure, but we're not using abx to evaluate recording equipment here, but encoders. Even if we were, the same problem arises: what alternative test, that can rule out guessing and bias and placebo, are you suggesting should be used? (let alone one that can pick up finer differences than abx!). Where are your pliers?

You can use abx to measure the quality of recording equipment as well, but the difficulty is in repeating the same source sound identically from test to test. The problem with using a performance as the source is not any problem with abx testing in principle but rather the impossibility of reproducing a performance twice (identically) AND the subtlety of what one is trying to detect (exactly because there is EXTREME faithfulness of encoders to originals--they're nearly identical). (If you were measuring something very gross relative to difference in performance, an abx test would easily be possible with a performance as the source; in such a case the minor variations from one performance to the next would not matter.) That's why you have e.g. tuning forks -- they isolate exactly what you want to test to a specific sound (thereby making it a "gross" factor, not small in a mass of other factors) and also can reproduce the same sound repeatedly.

There are circumstances where no good scientific test is possible, but that's a defect, not a recommendation, particularly when the differences get subtle and bias and confabulation become far more likely.

p.s.  I can't resist adding this beautiful example by Feynman of the power of bias.  The only thing I would add is that it is now and will ALWAYS be a problem, even in the sciences, because sometimes there are very powerful desires behind a bias, and scientists are by no means immune:
Quote
We have learned a lot from experience about how to handle some of the ways we fool ourselves. One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be quite right. It's a little bit off because he had the incorrect value for the viscosity of air. It's interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher. Why didn't they discover the new number was higher right away? It's a thing that scientists are ashamed of--this history--because it's apparent that people did things like this: When they got a number that was too high above Millikan's, they thought something must be wrong--and they would look for and find a reason why something might be wrong. When they got a number close to Millikan's value they didn't look so hard. And so they eliminated the numbers that were too far off, and did other things like that. We've learned those tricks nowadays, and now we don't have that kind of a disease.

ABX testing vs. perceptual states

Reply #28 –
Quote from: John Eppstein on
It's not that ABX testing, used correctly, is invalid. The key phrase is "used correctly". Some people appear to believe that ABX is some kind of panacea - that if it can't be proved by ABX it doesn't exist and that is simply not true.


Point us to one.


ABX testing vs. perceptual states

Reply #29 –
Quote from: John Eppstein on
Quote from: TechVsLife on
The "source" is not the performance.  The source is the recording.

No. Not when we're evaluating recording equipment and what we're looking at is the cumulative effect that the recording chain and process has on the resulting recording. The recording is not the source, it is the product.

And here we have the essence of one of the problems with Ethan's paper and with your whole mindset - you're looking at gear used for recording the same way that you evaluate gear used for hi-fi playback, but it's not really the same thing. The tools are similar in "appearance" and perhaps even in the gross details of operation - but not in use. It's like comparing dentist's pliers used to pull teeth to fine jeweler's pliers used in the fabrication of fine filigree. Yes they both belong to the general tool category of "pliers" - but the way they are used and the result of the use is vastly different. One is constructive and intended for very detailed assembly. The other assembles nothing and, if anything, is destructive in function, albeit beneficial.


And what, you couldn't have mentioned this 2 pages ago?
Sorry, but this just sounds like you're trying to move the goal posts to make yourself sound right. Not to mention the impossibility of ABXing a performance.

Your post is rife with inconsistencies anyway.

- If you are listening back to a recorded performance in the studio, you *are* listening to "hi-fi playback"
- The recording is always the source, when you are listening to it.
- who said anyone here was looking at recording gear as being exactly the same as a consumer hi-fi?
"...effect that the recording chain and process has on the resulting recording"
- so...what... you listening to a recording or not?

And, what the hell have plliers got to do with audio? And yes, I know it's an analogy. It's just that it's not a useful one.

ABX testing vs. perceptual states

Reply #30 –
If I may address all, please, folks, realize that the creative side and the technical side are both important in this industry, and it woudl be remarkably nice, not to say unusual and amazing, if we could get along with each other once in a while.

John, ABX testing really does work for determining if a given subject can hear a difference, when it's done right.  Nobody can prove a negative, not a scientist or anyone else, but there is valid statistical inference regarding populations from repeated test results, being negative or positive. There is never any absolute, but a probability suggesting that the age of the universe is involved is certainly definitive.

Techie guys, you know, the artistic side tries to lay down what sounds good. Euphony is their stock and trade. There's more than a little bit to be gained in the knowlege they have, as well, rather than simply telling them that their tape deck distorts and isn't as accurate. It's NOT as accurate, but it might just sound better, since rather obviously, what we capture in the studio or anywhere else is about, oh, what, 1/10000th of the analytic information in the soundfield around the listener's head.


So, how about some friendly dialog?

Hydrogen audio may not be the best place, I suppose, you know, you CAN accurately measure a given person's preference, but with an ABC/hr test, so strictly speaking, it wouldn't be ABX, it would be another test.  So, I suppose purely testing prefereince would violate TOS 8.

But given the ranting I get from audiopiles (check out the talk announcement I put up on stereophile board), I understand why TOS 8 exists. Some arguments just aren't worth having.

So, dunno an answer, but fighting accomplishes nothing.
-----
J. D. (jj) Johnston

ABX testing vs. perceptual states

Reply #31 –
Quote from: Woodinville on
Techie guys, you know, the artistic side tries to lay down what sounds good. Euphony is their stock and trade. There's more than a little bit to be gained in the knowlege they have, as well, rather than simply telling them that their tape deck distorts and isn't as accurate. It's NOT as accurate, but it might just sound better, since rather obviously, what we capture in the studio or anywhere else is about, oh, what, 1/10000th of the analytic information in the soundfield around the listener's head.


So, how about some friendly dialog


Thank you, you nailed it.

My previous post in another thread was all about building a bridge between the tech side  and the aristic side. It got toasted, and I hope this one will stay up.

1) about abx: I agree with Woodinville, the limitations doesn't invalidate the test. But is has to be done right. I have seen that a 0.1 dB difference in level can yield to the perception that file A sounds better than file B, even if everyone feels that they are exactly the same level.

2) about accuracy: accuracy over what ? the source ? the electrical signal comming out of the mike ? In any case, it will not be accurate to what is going on in the tracking room, digital, analog, a soundblaster card or a Lavry Gold converter. It's like aguing that a photo is more accurate than a painting: it just doesn't makes sense, both are inacurate. So recording a band is an art, and as an artist, I use all means necessary, any tools, that makes my recordings closer to what is happening in the room. And sometimes, accuracy is not what does the trick, sometimes, an "inacurate" studer A-800 is closer to the live band's energy and "balls".

3) so this yield to this: Professional recordists, AE, ME, or producer, are NOT audiophiles. They buy the gear they feel are making their job easier, and if bias is making them chose an expensive discrete class A EQ over a pluggin, it doesn't necessary means they are wrong about their choice.

4) Now about perceptual state of the brain, while fascinating, I'm not sure it invalidates the abx test. But maybe it can be useful to study how a trained human being can perceive things that others don't.

Hope this post stays up, I don't think it is really violating any TOS rule.

wow, tough crowd you have here

malice

ABX testing vs. perceptual states

Reply #32 –
Quote from: Woodinville on
Hydrogen audio may not be the best place, I suppose, you know, you CAN accurately measure a given person's preference, but with an ABC/hr test, so strictly speaking, it wouldn't be ABX, it would be another test.  So, I suppose purely testing prefereince would violate TOS 8.

Please read the TOS first. It specifically mentions ABC/HR and most larger codec tests here have been done with ABC/HR AFAIK.

The problem is that the kind of objections made here to ABX (those of mental state and not being able to prove a negative) tests do not go away with ABC/HR.
"We cannot win against obsession. They care, we don't. They win."

ABX testing vs. perceptual states

Reply #33 –
Quote from: John Eppstein on
Quote from: TechVsLife on
The "source" is not the performance.  The source is the recording.

No. Not when we're evaluating recording equipment and what we're looking at is the cumulative effect that the recording chain and process has on the resulting recording. The recording is not the source, it is the product.


So John is what you're saying is that the same basic kinds of performance and orthodox electronic theory that make a digital convertor good for playback does not make it good for recording?

We already know that you seem to believe that recording gear is susceptible to a mysterious effect that is impossible to model with orthodox mathematics called "stacking".

We already know that you seem to believe that desptite the fact that it is widely used in digital recording equipment and software, the Fourier Transform just sort of stops working when you get it too close to the microphone and performer.

We know that you seem to believe that orthodox audio concepts such as noise and distortion don't apply to analog recorders if their make and model is "legendary" as in Studer.

I'm thinking about the reciprocity effect which says that every passive acoustical element behaves symmetrically. You don't believe that reciprocity applies to recording, either?

And then there is ABX and all the psychoacoustic, psychological, and social effects that it addresses (AKA Bias) for playback euqipment. You're saying that applying that same knowlege to evaluating the recording chain is similarly ill-advised?  Recordists are invulnerable to bias?

Not so much! ;-)

Quote
And here we have the essence of one of the problems with Ethan's paper and with your whole mindset - you're looking at gear used for recording the same way that you evaluate gear used for hi-fi playback, but it's not really the same thing.


Your first problem John is that Ethan and I are recordists. I know that Ethan records well because I've heard his work. The recordings I make are copyrighed by a number of someone else's so I can't freely disseminate my work like Ethan does. And, they are not made using professional artists in professional grade venues. So I don't have that to wave around. I can fearlessly predict that other than an outright miracle of God, I'll never be credited with anything to do with an award-winning recording! However, I probably have in the past 10 years made more recordings by accident than most professional recordists have made on purpose.

So John, you have absolutely no God-given right to talk down to Ethan and I. If you don't believe in God, subsitute legal, moral, ethical, or techniacl for God. If you want to talk to us as a peer, we'll consider that despite all of the egregious false technical claims of yours that we have had to correct in the past 5 days.

Quote
The tools are similar in "appearance" and perhaps even in the gross details of operation - but not in use.


So John it is your claim that electrons that are part of the recording process are somehow different than electons that are part of the playback process?  How does that work when I hook moitoring equipment from a playback system up to a recording rig? Does good equipment suddently become bad? Does bad equipment suddenly become good? Do accurate speakers suddenly stop being accurate because of the direct copper/silicon/mylar/aluminum connection to the mic preamp  and a microphone? Does a CD player somehow cleanse signals of recording-system cooties?

Not so much! ;-)

Quote
It's like comparing dentist's pliers used to pull teeth to fine jeweler's pliers used in the fabrication of fine filigree. Yes they both belong to the general tool category of "pliers" - but the way they are used and the result of the use is vastly different. One is constructive and intended for very detailed assembly.


Joh, you are picking the wrong pliers. Try the pliers that orthodontists use to adjust braces. Your mistake here is that you are comparing a brute force tool to a fine adjustment tool, which invlidiates your comparison of dentisty to jewelry making.

It turns out that dentistry and jewelry-making share many tools and processes. For example lost-wax casting is used to make both jewelry and dental prosthesis such as metalic crowns and bridges.  I often see jewelers using dental picks and hemostats. Dental hand grinders and bits are often used for carving  and polishing jewelry. The biggest difference is that dental tools are more frequently either plated or made of stainless steel, and they are always sterilized before use. Some dentists even make jewelry using dental equipment as a hobby.

ABX testing vs. perceptual states

Reply #34 –
Quote from: Arnold B. Krueger on
Recordists are invulnerable to bias?

Not so much! ;-)


I don't believe recordists are invulnerable to bias (as I don't share many of John's view, mind you)

I do believe that recordists (and I mean professional experienced recordists) have one major advantage as matter as bias is concerned. Good engineer mixers (especially mixers) know that bias is the number one concern in their field. Mixers do know that the mind is a very tricky thing that will make their mixes sound good while they are "not so good yet". Same applies to ME and  I think it's Bob Katz that said once : "you will only have one chance to hear a mix for the first time". I would be very curious to learn more about a serious research about this topic of bias and how it affect our judgement and how we integrate the fact that we KNOW we are vumlnerable.

Quote
Your first problem John is that Ethan and I are recordists. I know that Ethan records well because I've heard his work. The recordings I make are copyrighed by a number of someone else's so I can't freely disseminate my work like Ethan does. And, they are not made using professional artists in professional grade venues. So I don't have that to wave around. I can fearlessly predict that other than an outright miracle of God, I'll never be credited with anything to do with an award-winning recording! However, I probably have in the past 10 years made more recordings by accident than most professional recordists have made on purpose.


I do not know about your credentials in matter of recordings, and I certainly won't evoke what I know about Etahn's work.

I do wish you don't take this the wrong way, but Engineers/techs that are mastering BOTH fields and are award winning mixers are extremely rare. Massenburg, Putnam comes to mind. You still can find younger dudes having both the set of skills: Dave Collins, my favorite Mastering Engineer is responsible of many designs and I quite admire him for that.

You did notice I was not good enough in the Electrical engineering side. And, I might be wrong, and I hope you'll forgive me about this, but I've seen comments from you that tells me you would be able to handle a very long argument about mixing with me or mixerman.

Neither of us are "George Massenburg", I hope you realise that what I'm being humble here. I do have a lot of things to learn from tech/engineer guys. And my point is that you should hear what someone like Mixerman has to say. I can assure you he doesn't know shit about how a preamp works, but he IS a heavyweight in  the more general field that is "Proffessional audio recording"

I do hope you understand and see the open hand here.

Because when I read someone like JJ understanding my craft like he does at the womb forums, I feel the urge to be very attentive to what he has to teach me about his knowledge.

peace


malice
 

ABX testing vs. perceptual states

Reply #35 –
Quote from: malice on
Quote from: Arnold B. Krueger on
Recordists are invulnerable to bias?

Not so much! ;-)


I don't believe recordists are invulnerable to bias (as I don't share many of John's view, mind you)

I do believe that recordists (and I mean professional experienced recordists) have one major advantage as matter as bias is concerned. Good engineer mixers (especially mixers) know that bias is the number one concern in their field. Mixers do know that the mind is a very tricky thing that will make their mixes sound good while they are "not so good yet". Same applies to ME and  I think it's Bob Katz that said once : "you will only have one chance to hear a mix for the first time". I would be very curious to learn more about a serious research about this topic of bias and how it affect our judgement and how we integrate the fact that we KNOW we are vumlnerable.


It turns out that Katz and Massenberg have granted crediblity to blind listening tests in public statements. I know of no hand-waving from them about fear and loathing and performance anxiety.

Quote
Quote
Your first problem John is that Ethan and I are recordists. I know that Ethan records well because I've heard his work. The recordings I make are copyrighed by a number of someone else's so I can't freely disseminate my work like Ethan does. And, they are not made using professional artists in professional grade venues. So I don't have that to wave around. I can fearlessly predict that other than an outright miracle of God, I'll never be credited with anything to do with an award-winning recording! However, I probably have in the past 10 years made more recordings by accident than most professional recordists have made on purpose.


I do not know about your credentials in matter of recordings, and I certainly won't evoke what I know about Ethan's work.


I don't know about yours Malice. I asked you to provide your credentials privately several days back and you are a no-show.

As far as Ethan's work goes, it is a jillion times better than what is available to me of yours, because something  that is even just halfways-decent always beats absolutely nothing.

Quote
You did notice I was not good enough in the Electrical engineering side.


To say the very least!  This raises further issues about crediblity credibility because you did give your statements as if you thought you were some kind of authority in the matter. "I don't know" is a fair answer that gets respect. So is silence.

Quote
I might be wrong, and I hope you'll forgive me about this, but I've seen comments from you that tells me you would be able to handle a very long argument about mixing with me or mixerman.


That  statement is  trash talk because you have provided no credentials, despite you making an issue of the matter and failing to provide them, even privately.  The womb has proven itself to be far from being a paragon of free speech.



ABX testing vs. perceptual states

Reply #36 –
Quote from: Woodinville on
If I may address all, please, folks, realize that the creative side and the technical side are both important in this industry, and it woudl be remarkably nice, not to say unusual and amazing, if we could get along with each other once in a while.

[snip]

So, dunno an answer, but fighting accomplishes nothing.
Exactly. I see that this thread is already starting to get dragged off-topic over the same personality conflicts. I don't like seeing this, and if it continues, I will put my foot down and moderate as firmly as in the AES thread.

ABX testing vs. perceptual states

Reply #37 –
Quote from: Canar on
Quote from: Woodinville on
If I may address all, please, folks, realize that the creative side and the technical side are both important in this industry, and it woudl be remarkably nice, not to say unusual and amazing, if we could get along with each other once in a while.

[snip]

So, dunno an answer, but fighting accomplishes nothing.
Exactly. I see that this thread is already starting to get dragged off-topic over the same personality conflicts. I don't like seeing this, and if it continues, I will put my foot down and moderate as firmly as in the AES thread.


No, I do think we're on the right track.

I'm genuinly trying to extand a hand here.

I apreciated the PM Ethan sent me and I'm willing to put my energy into calming this discussion both here and at the Womb.

I'll address Mr Krueger privately and refrain from fueling any dispute.

And I do think we can learn from all this.

Peace

malice




ABX testing vs. perceptual states

Reply #38 –
Maybe I've missed the point, but I don't see why the discussion has to be rancorous.  It's certainly possible that abx testing or the other tests that are well-accepted here do not cover all forms of listening perception or even interfere with those other forms of perception.  But what is the way of proving the existence of that other form of perception?  What is the alternative testing to be used? 

It's as if one were identifying a forgery or a wine knockoff from the originals.  Someone may do it through meticulous concentration on brushstrokes or acidity.  Someone else react to the whole, and say the painting or wine lacks depth, warmth, richness etc.  Fine.  There's no problem either way, whatever perceptive capacity they exercise, if they can consistently identify the painting or wine without cheating. 

All the tests used in this forum are designed to rule out guessing and cheating (and the worst kind is the unknowing or unintentional one of bias).  I don't see any particular theory required by any of the tests, except for commonplace observations about the prevalence of bias, mechanisms to avoid or neutralize that bias, and the use of well-established laws of statistics.   

So why any rancor?  There's no attachment to a theory or not much of one, and no attachment even to a particular test, if there's another test that can rule out bias and guessing. 

ABX testing vs. perceptual states

Reply #40 –
Quote from: TechVsLife on
It's certainly possible that abx testing or the other tests that are well-accepted here do not cover all forms of listening perception or even interfere with those other forms of perception.  But what is the way of proving the existence of that other form of perception?  What is the alternative testing to be used?

Indeed. It is possible that ABX tests do not cover all forms of listening perception. Russell's Teapot is worth considering here, even if it was initially proposed in a different context:

Quote
If I were to suggest that between the Earth and Mars there is a china teapot revolving about the sun in an elliptical orbit, nobody would be able to disprove my assertion provided I were careful to add that the teapot is too small to be revealed even by our most powerful telescopes. But if I were to go on to say that, since my assertion cannot be disproved, it is an intolerable presumption on the part of human reason to doubt it, I should rightly be thought to be talking nonsense.

If not ABX then what? What would these tests look like? ABX-detractors do not go so far as to identify alternate procedures that contain adequate safeguards against the critical errors caused through placebo and sighted-testing. They do not do so for one simple reason: They lack the nuanced understanding of statistics and science required to create such a novel methodology.

Double-blind tests are one of the gold standards of scientific testing. Where they cannot be undertaken, researchers frequently complain that they wish they could be.

ABX testing vs. perceptual states

Reply #41 –
Quote from: TechVsLife on
...But what is the way of proving the existence of that other form of perception?  What is the alternative testing to be used?...

You can't prove if it doesn't exists.
I don't know why some people here don't understand that brain hears what comes through the ears and than imagine or fabricates extra information according to ones biases and mental health. The ABX test eliminates that extra imagined "information". There are people out there that hear voices that nobody else can (schizophrenia). Does that mean they have a better hearing?!

It doesn't matter how euphonic your imagination is. It is irrelevant to anybody else because it is inside your head only. It doesn't exist. That's why it would not survive a double blind test.

There is no such a thing as somebody that have a better discernment in a test that is not double blinded. The double blinded test gives you always a better discernment. Otherwise your "taste" and inclination to like more certain brands over others, added to, lets say, low confidence level, desire to blend in and other psychological deformations will drive you to choose what is more complex and more expensive instead of what actually makes sense.

ABX testing vs. perceptual states

Reply #42 –
Quote from: malice on
Quote from: Woodinville on
Techie guys, you know, the artistic side tries to lay down what sounds good. Euphony is their stock and trade. There's more than a little bit to be gained in the knowlege they have, as well, rather than simply telling them that their tape deck distorts and isn't as accurate. It's NOT as accurate, but it might just sound better, since rather obviously, what we capture in the studio or anywhere else is about, oh, what, 1/10000th of the analytic information in the soundfield around the listener's head.


So, how about some friendly dialog


Thank you, you nailed it.

My previous post in another thread was all about building a bridge between the tech side  and the aristic side. It got toasted, and I hope this one will stay up.

1) about abx: I agree with Woodinville, the limitations doesn't invalidate the test. But is has to be done right. I have seen that a 0.1 dB difference in level can yield to the perception that file A sounds better than file B, even if everyone feels that they are exactly the same level.



Well, 'feeling' they are at the same level isn't good enough.  Actually matching outputs (e.g., with a voltmeter*) is what's required.  If levels weren't properly matched, then a ABX that's positive for difference is not surprising.  ABX *works*. 


(* Level matching for DBT is typically specified to be within 0.2dB, so your result here is unusual in that regard. Was the program material test tones, or something with a lot of content in the range we're most sensitive to e.g. voice?)


Btw, referencing ABC/hr isn't a violation of TOS at HA.  I'm pretty sure they or something like them have been used for codec listening tests here.


ABX testing vs. perceptual states

Reply #43 –
Quote from: krabapple on
(* Level matching for DBT is typically specified to be within 0.2dB, so your result here is unusual in that regard. Was the program material test tones, or something with a lot of content in the range we're most sensitive to e.g. voice?)


I'm trying to remember where I read this, I'm positive it was a discussion with Bob Katz and some other professionals. I'll report links if I find them back.

That said I agree with you, it was just a remark to elaborate on the necessity of being extremely cautious with matching levels in tests.

malice

ABX testing vs. perceptual states

Reply #44 –
Quote from: Light-Fire on
the brain hears what comes through the ears and then imagines or fabricates extra information according to one's biases and mental health.
...
It doesn't matter how euphonic your imagination is. It is irrelevant to anybody else because it is inside your head only. It doesn't exist. That's why it would not survive a double blind test.

I agree with you that this is going on in the case of "audiophools."  But is it possible to hear anything without imagination?  It's hard to say what music is--it's evidently a lot more than air getting compressed with certain frequencies and amplitudes etc.  So it's possible only one person might understand a piece of music that no one else could--it might exist in his head only, but it would still exist.  (Of course he likely would be able to perceive or imagine it again if the same sounds are repeated, so some aspect of that could be captured in a double-blind test. And one would expect that this music would be partially understood/appreciated by other people.)

this is I admit pedantic, but I'd be wary of suggesting that we have access to what's objective or "in itself" without interpretation and imagination.  Everything we see or hear is already other than (though it must correspond somehow with) what it is before being seen or heard (I couldn't even call it "light" or "compressed air" because that's already after our minds have interpreted "things").  There's a limit on science -- it doesn't lift the basic darkness -- but we take what light we can get and for that it's helpful to see the extreme opposite points that expose the basic obscurity.

ABX testing vs. perceptual states

Reply #45 –
Quote from: Woodinville on
If I may address all, please, folks, realize that the creative side and the technical side are both important in this industry, and it woudl be remarkably nice, not to say unusual and amazing, if we could get along with each other once in a while.

John, ABX testing really does work for determining if a given subject can hear a difference, when it's done right.  Nobody can prove a negative, not a scientist or anyone else, but there is valid statistical inference regarding populations from repeated test results, being negative or positive. There is never any absolute, but a probability suggesting that the age of the universe is involved is certainly definitive.

Techie guys, you know, the artistic side tries to lay down what sounds good. Euphony is their stock and trade. There's more than a little bit to be gained in the knowlege they have, as well, rather than simply telling them that their tape deck distorts and isn't as accurate. It's NOT as accurate, but it might just sound better, since rather obviously, what we capture in the studio or anywhere else is about, oh, what, 1/10000th of the analytic information in the soundfield around the listener's head.

I "get" all this. I'd even happily extend it into the home listening environment: if someone thinks a certain kind of noise, distortion, artefact etc makes audio sound nicer to them, that's fine. Of course, if they claim it's better, they'll get torn to shreds on HA - because most people here define better = more accurate, rather than better = sounds nicer. But maybe we can avoid such cat-fights by sticking to clear terminology.

What "most" audiophiles and many people in pro-audio don't "get" is that not every piece of equipment has its own sonic signature. Some of it simply doesn't change the sound in any audible way - any changes are inaudible to every human on the planet, whatever source material and ancillary equipment is used.


What's even sillier is the criticism of double blind testing. All it means is that no one in the room should know what you're listening to. That's it. If you want people to believe you really hear a difference, identify which-is-which enough times that there's a small chance of being correct that many times by accident.

That first part - double blind testing = not knowing what you're listening to - is really really easy. You have to wonder why hi-fi magazines don't do it routinely.



Cheers,
David.

ABX testing vs. perceptual states

Reply #46 –
Ok, I did make some research about that 0.1 db increase, and I'm afraid I cannot yet post that post from Bob Katz as it is part of a private forum that forbids me to quote exactly or refeer about the sources.

But I can elaborate on the very interesting things that has been said and that shows how difficult it is to evaluate anything analog in tests like ABX. Not saying impossible, but you have to take in account what you are testing.

The first argument was about the drift of some analog devices shows, even with 20-turn trim posts, calibration can change, and 0.1 dB is not out of question. If you add a very slight of load, you can end up with another 0.1 dB and you're off the charts of proper abx testing.

The second argument is about the euphonic qualities that we are debating for several days.
It refeers to how some people, like myself, feel that small amount of distortion can increase  apparent depht and dimention, and therefore, sounds "better".

And that would be an impossible can of worms to debate here because it's impossible to know if that "better perception" is caused by the increase of the harmonic distoprtion components that increase the overal level or by a real "improvement" in depht and dimension caused by the said distortion.

So yes: very difficult to prove or disprove this but that is definitly worth considering when you compare a good tape recorder and a converter or an analog EQ with big Iron transformers in it, and a pluggin EQ.

malice

ABX testing vs. perceptual states

Reply #47 –
Quote from: 2Bdecided on
That first part - double blind testing = not knowing what you're listening to - is really really easy. You have to wonder why hi-fi magazines don't do it routinely.


Incidently, I wonder why I cannot find an hi-fi magazine or a pro-audio magazine totally unbiased because of  their sponsors.


malice

ABX testing vs. perceptual states

Reply #48 –
Quote from: malice on
Ok, I did make some research about that 0.1 db increase, and I'm afraid I cannot yet post that post from Bob Katz as it is part of a private forum that forbids me to quote exactly or refeer about the sources.


People can say whatever they want. I can say that I've heard 0.01 dB level differences but oh darn, I've lost track of the actual details. ;-)

Back in the 80s there was a guy named Peter Moncrieff who claimed to hear differences measured in millibels. Strangley enough nobody was ever able to duplicate his work in a bias controlled listening test. ;-)

Quote
But I can elaborate on the very interesting things that has been said and that shows how difficult it is to evaluate anything analog in tests like ABX. Not saying impossible, but you have to take in account what you are testing.

The first argument was about the drift of some analog devices shows, even with 20-turn trim posts, calibration can change, and 0.1 dB is not out of question. If you add a very slight of load, you can end up with another 0.1 dB and you're off the charts of proper abx testing.


I've been level-matching all sorts of audio equipment, even relatively unstable equipment such as analog tape and vinyl playback, since the late 1970s. Analog tape and vinyl can definately drift or wobble during playback by up to 1  dB. Normal signal-handling equipment such as amplifeirs, preamps, equalizers and the like have no problem staying within 0.1 dB.

Very shortly after we started doing ABX, I developed a precision level-matching device that was composed of  a pair of 10-turn potentiometers. I put crank handles on the pot shafts, and it was called "The Fishing Reels". It was basically overkill because we ended up adjusting it with big moves. I've had no problems setting levels within 0.1 dB using  RN50 single turn linear taper potentiometers.

Quote
The second argument is about the euphonic qualities that we are debating for several days.
It refeers to how some people, like myself, feel that small amount of distortion can increase  apparent depht and dimention, and therefore, sounds "better".


As soon as there is an actual audible difference, then how it perceived needs to be evaluated on a individual basis. In some cases ditortion can be added to compensate for an undesirable distortion that crept in from someplace else. What a given kind of distortion sounds like is usually very dependent on the actual music you listen to through it.

Quote
And that would be an impossible can of worms to debate here because it's impossible to know if that "better perception" is caused by the increase of the harmonic distoprtion components that increase the overal level or by a real "improvement" in depht and dimension caused by the said distortion.


Here we see yet another example of an unfortunate choice of words that may lead to false beliefs.

There is no such thing as harmonic distortion as a property of equipment.

Harmonic distortion is an artificial theoretical concept that is used to characterize or measure nonlinear distortion.

All nonlinear distortion creates both harmonics and intermodulation products. However, low pass filtering can prevent harmonics from ever being measured. 

Nobody can ever increase harmonic distortion without also increasing intermodulation distortion. In fact it is possible for equipment to be nonlinear and produce no measurable harmonic distortion, but it will still produce measurable IM. 

For example,  back in the days when resistive networks were used to build DACs there could be signficiant nonlineariities in the resistive network but the reconstruction filter would strip off the harmonics for test signals above about 6.5 KHz. This would create an apparent exception to the usual rule of nonlinearity in electronics increasing at the highest frrequencies. 

Intermodulation distortion produces non-harmonic or aharmonic distoriton as it acts on real-world music, which is more likely as being perceived as being undesirable because it is not musical.

ABX testing vs. perceptual states

Reply #49 –
Quote from: malice on
Incidently, I wonder why I cannot find an hi-fi magazine or a pro-audio magazine totally unbiased because of  their sponsors.
There's at least one on-line hi-fi "magazine" which is subscription only, and carries no adverts. It claims to be unbiassed. (If I could remember what it was called, I'd say - I'm being forgetful, not coy!)

There's Which? magazine in the UK, and (is it called) consumer reports in the USA - I don't believe either of these is biassed due to adverts, but I don't know if their testing counts for much.

Cheers,
David.