From the article (http://www.stereophile.com/content/music-matters-6-seattle-wednesday) on Stereophile.com:
Appearing in person and each giving presentations lasting 30-minute, will be representatives from Audio Research, Ayre Acoustics, B&W, Classé, Finite Elemente, GoldenEar, Harmonic Resolution, Linn, Meridian, Peachtree Audio, Transparent, and Wilson Audio, while Stereophile editor John Atkinson will be demonstrating the benefits of high-resolution audio and the evils of MP3, using the master files of some of his recordings.
It's this Wednesday, Feb. 9, and while I live about 25 minutes away from Definitive's Seattle showroom, I think I'll manage to avoid that - but what am I saying! Of *course* the MP3 vs. high-rez demo will be ABX'ed...
From the article (http://www.stereophile.com/content/music-matters-6-seattle-wednesday) on Stereophile.com:
Appearing in person and each giving presentations lasting 30-minute, will be representatives from Audio Research, Ayre Acoustics, B&W, Classé, Finite Elemente, GoldenEar, Harmonic Resolution, Linn, Meridian, Peachtree Audio, Transparent, and Wilson Audio, while Stereophile editor John Atkinson will be demonstrating the benefits of high-resolution audio and the evils of MP3, using the master files of some of his recordings.
It's this Wednesday, Feb. 9, and while I live about 25 minutes away from Definitive's Seattle showroom, I think I'll manage to avoid that - but what am I saying! Of *course* the MP3 vs. high-rez demo will be ABX'ed...
Atkinson posts here sometimes. Maybe he'll tell us what means will be used to 'demonstrate' the 'evils' of MP3.
My guess is it'll once again be along the lines of his previous 'demonstration' a la Massenburg: subtract and play the difference signal and let the listeners say OOH HOW BAD.
With the right samples it won't be too hard to make mp3 look bad.
I find it reassuring that people still go to events to share their passion for music reproduction
Surprisingly the invitation doesn't contain a single occurrence of the terms "analog", "vinyl", "turntable" or "tube" but plenty of "digital". Could that be a trend ?
The "demonstrating the benefits of high-resolution audio" is interesting, since a quick check of the presented monitors datasheets indicates that none of them seems to go far beyond 20kHz.
"Light hors d'oeuvres & refreshments" . . . sounds like a multisensory experience !
Could that be a trend ?
Fashion and trend is all they have.
Could that be a trend ?
Fashion and trend is all they have.
I'd be more interested in his demonstration on the evils of transcoding from lossy mp3 to FLAC or mp3 to AAC LOL
Surprisingly the invitation doesn't contain a single occurrence of the terms "analog", "vinyl", "turntable" or "tube" but plenty of "digital". Could that be a trend ?
Mmm, perhaps, but more likely just Definitive's SOP.
AFAIK, the only brand of tubed equipment they carry is Audio Research, and while they do carry some turntables, vinyl has never really figured heavily in their demo systems.
Their "big room" is more likely to have a Levinson Reference front end hooked up to a pair of Levinson's current monster monoblocks driving a pair of Wilsons, B&Ws, or Theils, than a tube pre and low-powered SET driving a pair of horn-loaded beasts (although I see that they are carrying Magico, albeit in only one of their showrooms).
High-end home theater is a big deal for them, and that's definitely a digital world...
I find it reassuring that people still go to events to share their passion for music reproduction
...and if that passion were tempered with common sense and a willingness to actually understand and learn how various technologies work, I would agree - but sadly, that rarely seems to be the case.
While this event might not have the best track record with regard to objectivity - this has been discussed in the past - by all means it can make sense and entertain to audition high end speaker systems. And also, while all amps above a certain (moderate) quality level should sound the same, the high end audio field has to offer beautiful engineering work and some people find delight in that. Just like sports cars with enormous engines are being sold in a country with measly speed limits of 80 mph at max. Well, it gives you an "accelerative edge" (while commuting between 30-60 mph). It's not much different when you buy uber-monoblocks to drive your 10-50000 +/- 1dB speakers with them.
wat, satanic messages in my MP3s?
wat, satanic messages in my MP3s?
Ask the RIAA. They will tell you that "MP3 = EVIL".
Ask the RIAA. They will tell you that "MP3 = EVIL".
That's ridiculous. M-P-3 is
three letters. E-V-I-L is
four letters.
Silly R-I-A-A.
- M.
Atkinson is certainly one of the last people on earth I'd choose to explain or demonstrate ANYTHING about audio.
Atkinson is certainly one of the last people on earth I'd choose to explain or demonstrate ANYTHING about audio.
Ad hominem is certainly one of the last things on Earth I'd like to see regarding an event such as this. If you want to disprove something he's said, go right ahead. But please play nice.
Atkinson is certainly one of the last people on earth I'd choose to explain or demonstrate ANYTHING about audio.
Ad hominem is certainly one of the last things on Earth I'd like to see regarding an event such as this. If you want to disprove something he's said, go right ahead. But please play nice.
Mea culpa. I let my personal experience with this individual influence my post.
Atkinson is certainly one of the last people on earth I'd choose to explain or demonstrate ANYTHING about audio.
Ad hominem is certainly one of the last things on Earth I'd like to see regarding an event such as this. If you want to disprove something he's said, go right ahead. But please play nice.
Mea culpa. I let my personal experience with this individual influence my post.
I am not sure to what you are referring, Mr. Edwards, but my apologies if I have offended you.
And while my presentation in Seattle this evening will indeed not involve any ABX or forced-choice tests, I am sure that people will find it interesting.
After all, there's no substitute for personal experience. And who could object to an event that promotes the idea of listening to recordings with as high a quality as possible?
John Atkinson
Editor, Stereophile
I object to any presentation of lossy encoding that doesn't properly describe the mechanism by which it works and demonstrate how one goes about properly determining whether it works or does not work.
So far you have really only demonstrated to the objective-minded audio community that you are willfully ignorant of lossy encoding, unwilling to improve upon what is a clear deficiency in your presentation of the subject matter as pointed out by people who clearly know more about the subject than you do, and that you are knowingly misleading people. Personally, I find this quite shameful if not morally reprehensible.
Unless you plan on changing your behavior, John Atkinson, offering apologies for being offensive is nothing short of insincere.
my presentation in Seattle this evening will indeed not involve any ABX or forced-choice tests
That's a relief. I hate those people who demand evidence for assertions!
After all, there's no substitute for personal experience.
Anecdotes are no substitute for objective, reproducible evidence.
And who could object to an even that promotes the idea of listening to recordings with as high a quality as possible?
If the superiority of such quality to other (and probably less expensive) listening setups were proven by objective means, sure!
I object to any presentation of lossy encoding that doesn't properly describe the mechanism by which it works and demonstrate how one goes about properly determining whether it works or does not work.
That's a little too much to get into a 30-minute presentation, I am afraid. But even if people are skeptical of what I might say, I still believe they will enjoy the quality of the hi-rez audio files I will be playing. I will be using a system with a total cost of <$4k, in order to emphasize that you don't need megabucks systems to appreciate the benefits.
So far you have really only demonstrated to the objective-minded audio community that you are willfully ignorant of lossy encoding, unwilling to improve upon what is a clear deficiency in your presentation of the subject matter as pointed out by people who clearly know more about the subject than you do, and that you are knowingly misleading people. Personally, I find this quite shameful if not morally reprehensible.
Sorry to hear that. even if you were correct, I can only use the pulpits I have available to talk about what I believe to be true. Just as you are doing here.
Unless you plan on changing your behavior, John Atkinson, offering apologies for being offensive is nothing short of insincere.
Sorry you feel that way. I wasn't aware that I had done anything personally to Carl Edwards, as he claimed.
John Atkinson
Editor, Stereophile
I think Carl Edwards simply questions your credibility. From what I have read from you, I think he's quite justified in doing so.
30 minutes is more than enough time to present lossy encoding to the layman audio enthusiast in an honest way. Perhaps it's not enough time if you also want to satisfy your agenda convincingly which still appears to be at odds with an honest presentation.
And while my presentation in Seattle this evening will indeed not involve any ABX or forced-choice tests, I am sure that people will find it interesting.
John, that might depend on whether or not you finally get around to doing some well-implemented listening tests and other demonstrations.
After all, there's no substitute for personal experience.
IME many personal experiences are a total waste of my time. For example, sighted evaluations of lossy decoders operating as they are typically used to do high quality distribution of audio files are a complete waste of the time of anybody with serious intents.
And who could object to an event that promotes the idea of listening to recordings with as high a quality as possible?
Does one have to leave the comfort of one's own home to listen to such a thing?
And who could object to an event that promotes the idea of listening to recordings with as high a quality as possible?
Does one have to leave the comfort of one's own home to listen to such a thing?
Only to visit the shops, but rest assured that you'll be sufficiently served by "a system with a total cost of <$4k", so "you don't need megabucks systems to appreciate the benefits."
And who could object to an event that promotes the idea of listening to recordings with as high a quality as possible?
Does one have to leave the comfort of one's own home to listen to such a thing?
They do if they wish to hear the hi-rez master files of my own commercial recordings, Mr. Krueger, which is one of the two things mentioned in the promotion for the event.
If anyone from HA is going to be attending the Definitive Audio event this evening, I'd be happy to discuss anything in my presentations with them.
John Atkinson
Editor, Stereophile
Only to visit the shops, but rest assured that you'll be sufficiently served by "a system with a total cost of <$4k", so "you don't need megabucks systems to appreciate the benefits."
Am I correct in interpreting this as a perpetuation of the unproven myth that someone needs to spend four figures in order to hear the deficiencies in lossy encoding?
I hope it wasn't unclear that I was quoting Mr Atkinson?
They do if they wish to hear the hi-rez master files of my own commercial recordings, Mr. Krueger, which is one of the two things mentioned in the promotion for the event.
I'm sure they would be able to derive the exact same experience if our hi-re"z" master files were presented in high-bitrate lossy or properly converted to CDDA, assuming you don't first seed them with expectation bias, of course. It's already been suggested that the equipment might not capable of delivering frequencies much beyond those limited by CDDA (to say the least of the frequency response of your participants; we already know that CDDA can deliver higher frequencies than what you can personally hear, Mr. Atkinson).
What will be done to ensure that the listening environment will quiet enough so that the participants can hear anything below -96dBFS? I hope you're taking them in one at a time as to minimize audience noise. Wait, I forgot, you only have 30 minutes.
I hope it wasn't unclear that I was quoting Mr Atkinson?
You were clear; it was I who wasn't.
Please allow me to rephrase...
Am I correct in interpreting this as a perpetuation of the unproven myth on the part of John Atkinson that someone needs to spend four figures in order to hear the deficiencies in lossy encoding?
Only to visit the shops, but rest assured that you'll be sufficiently served by "a system with a total cost of <$4k", so "you don't need megabucks systems to appreciate the benefits."
Am I correct in interpreting this as a perpetuation of the unproven myth that someone needs to spend four figures in order to hear the deficiencies in lossy encoding?
Maybe I'm missing something, but where did he say that? AFAIK, he said only that you can hear it on systems under $4k. Aren't there ABX tests here and elsewhere that demonstrate the audibility of lossy encoding on fairly inexpensive equipment, e.g., headphones? In fact, I recall an interesting discussion here to the effect that under certain circumstances, loudspeakers with poor frequency response make lossy encoding more rather than less audible by interfering with psychoacoustic masking.
He didn't say that, at least not directly and perhaps not at all. Nevertheless, I thought it was a provocative and worthwhile question.
He didn't say that, at least not directly and perhaps not at all. Nevertheless, I thought it was a provocative and worthwhile question.
It's certainly an interesting question, and one with an answer that's at least partly counterintuitive -- at least, I didn't know about the relationship between the audibility of lossy compression and ragged frequency response until I read about it here. I think I was bothered by the fact that the question seemed to take JA's assertion, which is that you don't need expensive equipment to hear the difference, and transmute it into the opposite one, that you do. I guess I'm bothered by the personal nature of the entire thread. I don't know the details of the comparison so I don't know whether confirmation bias is a concern here. But if they're such as to be controversial, wouldn't it be more productive to ask John to post compressed and uncompressed excerpts so that people can ABX them in Foobar?
People on this forum have invested time urging John Atkinson be more objective in his presentation on the subject and he has essentially thumbed his nose at us as a response. All one needs to to is read the previous discussions and read his articles (not to mention articles he doesn't write but approves to be published) to get the impression that he is not the least bit interested in presenting the topic in an objective and unbiased way.
I suppose <$4k might be reasonable for a great home system assuming that the majority is spent on speakers, though one might be able to spend considerably less provided the speakers chosen are still adequate. Objective evidence that one needs to spend extra money on the equipment and cables to provide an analog source and drive these speakers beyond regular consumer-grade equipment in order to improve audible sound quality simply does not exist. I am not including money spent ensuring a great listening environment, and I doubt JA is either.
People on this forum have invested time urging John Atkinson be more objective in his presentation on the subject and he has essentially thumbed his nose at us as a response. All one needs to to is read the previous discussions and read his articles (not to mention articles he doesn't write but approves to be published) to get the impression that he is not the least bit interested in presenting the topic in an objective and unbiased way.
I suppose <$4k might be reasonable for a great home system assuming that the majority is spent on speakers, though one might be able to spend considerably less provided the speakers chosen were still adequate. Objective evidence that one needs to spend extra money on the equipment and cables to provide an analog source and drive these speakers beyond regular consumer-grade equipment in order to improve audible sound quality simply does not exist. I am not including money spent ensuring a great listening environment, and I doubt JA is either.
Hmmm . . . as far as I know, there's no objective evidence that esoteric cables improve audio quality; tests in the AES Journal indicate that they can affect frequency response, but the curves don't seem to be of a magnitude that would be audible in an AB test according to existing psychoacoustic data. However, since response errors are cumulative, it is possible that any response aberrations would be audible under limited circumstances. A very marginal improvement at best. There is objective evidence that amplifiers of a certain size are required to reproduce music at natural levels, depending of course on the efficiency of loudspeakers, room size, radiation pattern, and listening distance. So that sets a practical lower limit on amplifier cost, albeit one that will vary depending on circumstance. IIRC, op amps are audible on ABX tests; that might set an objective lower limit on the cost of audibly transparent electronics.
I'm not aware of any objective evidence that analog sources are generically superior to digital sources. On the contrary, objective evidence would seem to suggest that high quality digital sources are superior (high quality since ABX tests in the AES Journal and here suggest that under some circumstances, a 44.1 kHz sampling rate is audible, and it's been demonstrated that in some circumstances, 16 bits is inadequate to transparently reproduce the full dynamic range of acoustical music in the absence of noise shaping).
Alas, there doesn't seem to be a very good library of blind tests for converters and loudspeakers. Loudspeakers in particular are difficult for anyone without the resources of a manufacturer to compare in bind tests. It's possible to establish objective criteria to some extent, e.g., for bass extension and SPL, and these could potentially set a lower limit for the minimum cost of a loudspeaker. But how is one to judge, objectively, whether a $60,000 Magico is audibly superior to a $30,000 Wilson? Unlike cables or amplifiers, there's no controversy over the fact that loudspeakers sound different, but by the same token, we know from Floyd Toole's research that subjective judgments of loudspeaker quality are distorted by confirmation bias.
All of which leads me to think that your assertion is correct, give or take a few thousand dollars -- we have little or no objective evidence that spending more than $4000 will buy better sound -- but that there's also no objective evidence that spending more won't. My ears tell me that, in the case of loudspeakers anyway, spending more can, but of course my ears are affected by bias, so no firm conclusion is possible. I for one would love to see more blind tests in Stereophile, though I recognize that a meaningful testing regime could be difficult to arrange. (I also think that there are theoretical reasons to believe that some relatively costly loudspeaker technologies can have audible benefits, e.g., line sources have several advantages, multiple subwoofers produce reduce the effect of room modes, etc. These characteristics are measurable and sufficient psychoacoustic data exists to demonstrate objectively that they can have audible benefits. In other cases, measurements are easy but the psychoacoustic data is inadequate to draw firm conclusions.)
Only to visit the shops, but rest assured that you'll be sufficiently served by "a system with a total cost of <$4k", so "you don't need megabucks systems to appreciate the benefits."
Am I correct in interpreting this as a perpetuation of the unproven myth that someone needs to spend four figures in order to hear the deficiencies in lossy encoding?
Maybe I'm missing something, but where did he say that? AFAIK, he said only that you can hear it on systems under $4k.
That's correct. The system I used this evening cost $4200 (the amplifier-D/A converter was $300 more that I anticipated). I felt it fully up to the task of allowing listeners to appreciate what they were hearing. The context for my comment was the fact that the average price of a Stereophile reader's system is $15,000, BTW.
Was anyone there from HA? No-one identified themselves as such.
John Atkinson
Editor, Stereophile
Was anyone there from HA? No-one identified themselves as such.
What do you mean "from HA"? - This is a forum, I guess you'd understand the concept...
I'm not aware of any objective evidence that analog sources are generically superior to digital sources.
BTW, when I said analog source I meant whatever device provides the analog signal including DACs. My mistake.
Was anyone there from HA? No-one identified themselves as such.
What do you mean "from HA"? - This is a forum, I guess you'd understand the concept...
That was unnecessary. Not to take sides here but Mr. Atkinson asked a valid question considering that he offered for HA-members to take part in discussions at said event yesterday.
Was anyone there from HA? No-one identified themselves as such.
What do you mean "from HA"? - This is a forum, I guess you'd understand the concept...
That was unnecessary. Not to take sides here but Mr. Atkinson asked a valid question considering that he offered for HA-members to take part in discussions at said event yesterday.
Was it? The way I see it, he asked for people responsible for this forum (I'd guess, admin and maybe mods) - Not as an open invitation where he invited all participants in HA.
And that's my point - This is a forum where anyone with interest in audio can participate. That doesn't make a member any less than the admins that created the forum, at least in terms of said discussions.
Only to visit the shops, but rest assured that you'll be sufficiently served by "a system with a total cost of <$4k", so "you don't need megabucks systems to appreciate the benefits."
Am I correct in interpreting this as a perpetuation of the unproven myth that someone needs to spend four figures in order to hear the deficiencies in lossy encoding?
Maybe I'm missing something, but where did he say that? AFAIK, he said only that you can hear it on systems under $4k.
That's correct. The system I used this evening cost $4200 (the amplifier-D/A converter was $300 more that I anticipated). I felt it fully up to the task of allowing listeners to appreciate what they were hearing. The context for my comment was the fact that the average price of a Stereophile reader's system is $15,000, BTW.
Notwithstanding audiophile orthodoxy, the gear as specified above was irrelevant to the audience's take-home message compared to:
1) whether or not you told the listeners what format they were hearing
and
2) whether you used high or low lossy codec bitrates and good or poor lossy encoders, and whether or not you use 'killer' audio clips to demo
and
3) whether or not you performed a 'difference' demo and offered it as evidence of audible inferiority of mp3, without an explanation of how mp3 actually works
I wasn't there, and from your replies to date on this thread I can't glean what you actually did re points 1,2, and 3. Care to fill us in?
Btw, should you ever bring this road show to NYC, I will do my best to attend.
Was anyone there from HA? No-one identified themselves as such.
What do you mean "from HA"? - This is a forum, I guess you'd understand the concept...
That was unnecessary. Not to take sides here but Mr. Atkinson asked a valid question considering that he offered for HA-members to take part in discussions at said event yesterday.
Was it? The way I see it, he asked for people responsible for this forum (I'd guess, admin and maybe mods) - Not as an open invitation where he invited all participants in HA.
That's not how I interpreted it at all.
And that's my point - This is a forum where anyone with interest in audio can participate. That doesn't make a member any less than the admins that created the forum, at least in terms of said discussions.
I think you're over-interpreting. We're all 'from HA' to people from other forums.
Was anyone there from HA? No-one identified themselves as such.
What do you mean "from HA"? - This is a forum, I guess you'd understand the concept...
That was unnecessary. Not to take sides here but Mr. Atkinson asked a valid question considering that he offered for HA-members to take part in discussions at said event yesterday.
Was it? The way I see it, he asked for people responsible for this forum (I'd guess, admin and maybe mods) - Not as an open invitation where he invited all participants in HA.
To clarify, by "anyone. . . from HA" I meant literally _anyone_: members, guests, lurkers, moderators, etc. I had conversations with many people last night but if there was anyone who had been reading this thread, no-one said so.
When I next do a presentation like this, i will post the news well in advance to this forum in case any form HA will be able to attend.
John Atkinson
Editor, Stereophile
Notwithstanding audiophile orthodoxy, the gear as specified above was irrelevant to the audience's take-home message compared to:
1) whether or not you told the listeners what format they were hearing, and
2) whether you used high or low lossy codec bitrates and good or poor lossy encoders, and whether or not you use 'killer' audio clips to demo, and
3) whether or not you performed a 'difference' demo and offered it as evidence of audible inferiority of mp3, without an explanation of how mp3 actually works
I wasn't there, and from your replies to date on this thread I can't glean what you actually did re points 1,2, and 3. Care to fill us in?
I wasn't there either. I would be very interested in seeing Krabapple's questions answered. Do you seek to give listeners opportunity to draw their own conclusions? Do you seek to demonstrate best-of-breed audio encoders or is it primarily your intent to make listeners aware that encoders can produce audible artifacts? Do you do 'difference' demos on encoded audio?
To clarify, by "anyone. . . from HA" I meant literally _anyone_: members, guests, lurkers, moderators, etc. I had conversations with many people last night but if there was anyone who had been reading this thread, no-one said so.
When I next do a presentation like this, i will post the news well in advance to this forum in case any form HA will be able to attend.
Ok, then I'm sorry about that. I was just confused about why you wanted anyone to identify themselves as you mention here:
Was anyone there from HA? No-one identified themselves as such.
I was just confused about why you wanted anyone to identify themselves as you mention here:
"Was anyone there from HA? No-one identified themselves as such."
Because, as evidenced by the postings to this thread - and even the very existence of this thread - some people at HA have questions about what I do and say, and I welcome the opportunity to discuss things in person. For example, in 2005, following many confrontational postings Arny Krueger had made on Usenet, I invited him to debate me at Home Entertainment 2005, in order to clear the air.
John Atkinson
Editor, Stereophile
I wasn't there either. I would be very interested in seeing Krabapple's questions answered.
Had you been there - had _anyone_ from HA been there - you would have known what happened at my presentation. I am puzzled by the fact that there exists simultaneously much curiosity about my Seattle presentation but not enough curiosity for anyone to attend. The same phenomenon occurred when I gave a similar presentation in Colorado 2 years ago. As I said, the next time I give a presentation like this, I'll give HA plenty of notice in case someone wishes to and is able to attend.
John Atkinson
Editor, Stereophile
I wasn't there either. I would be very interested in seeing Krabapple's questions answered.
Had you been there - had _anyone_ from HA been there - you would have known what happened at my presentation. I am puzzled by the fact that there exists simultaneously much curiosity about my Seattle presentation but not enough curiosity for anyone to attend. The same phenomenon occurred when I gave a similar presentation in Colorado 2 years ago. As I said, the next time I give a presentation like this, I'll give HA plenty of notice in case someone wishes to and is able to attend.
I am in Colorado but was not on HA 2 years ago and did not hear about your presentation.
From reading this thread, I would say that there exists much hostility but not enough curiosity to compel HA members to attend. Unless you enjoy hostility, I'd be relieved that none of the posters in this thread came. Low curiosity because people here assume your presentation relies on some familiar methods which non-audiophiles believe adversely affect listener objectivity and audiophiles believe are essential to the pleasure of audio. That's what Krabapple was looking for in asking his questions. They're not difficult or trick questions. I'd still be interested in answers.
I wasn't there either. I would be very interested in seeing Krabapple's questions answered.
Had you been there - had _anyone_ from HA been there - you would have known what happened at my presentation. I am puzzled by the fact that there exists simultaneously much curiosity about my Seattle presentation but not enough curiosity for anyone to attend. The same phenomenon occurred when I gave a similar presentation in Colorado 2 years ago. As I said, the next time I give a presentation like this, I'll give HA plenty of notice in case someone wishes to and is able to attend.
John Atkinson
Editor, Stereophile
John,
Are there reasons for you not wanting to answer the questions regarding which encoder, bitrate, etc was used to create the mp3 files? You can see that if someone had an agenda against mp3 it would be fairly easy to demonstrate the short comings of mp3 at 96kbps, when in reality there is a world of difference between 96kbps and 320kbps.
The difference question is also pertinent.
I wasn't there either. I would be very interested in seeing Krabapple's questions answered.
Had you been there - had _anyone_ from HA been there - you would have known what happened at my presentation. I am puzzled by the fact that there exists simultaneously much curiosity about my Seattle presentation but not enough curiosity for anyone to attend. The same phenomenon occurred when I gave a similar presentation in Colorado 2 years ago. As I said, the next time I give a presentation like this, I'll give HA plenty of notice in case someone wishes to and is able to attend.
Are there reasons for you not wanting to answer the questions regarding which encoder, bitrate, etc was used to create the mp3 files?
Only that I did answer at length on HA the same questions, including those put to me by"krabapple," when I did the same demonstration in Colorado 2 years ago. See, for example, http://www.hydrogenaudio.org/forums/lofive...p/t71245-0.html (http://www.hydrogenaudio.org/forums/lofiversion/index.php/t71245-0.html) . See also http://www.hydrogenaudio.org/forums/index....72446&st=25 (http://www.hydrogenaudio.org/forums/index.php?showtopic=72446&st=25) .
You can see that if someone had an agenda against mp3 it would be fairly easy to demonstrate the short comings of mp3 at 96kbps, when in reality there is a world of difference between 96kbps and 320kbps.
Of course. But even 320kbps AAC has shown not to be transparent to all listeners under all circumstances, as appeared to be suggested at http://www.hydrogenaudio.org/forums/index....mp;#entry743201 (http://www.hydrogenaudio.org/forums/index.php?showtopic=85961&st=0&gopid=743201&#entry743201) . The musical examples I played in Seattle were the same as in Colorado: a single-blind presentation, perfectly level-matched, of 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1khz MP3 at 44.1kHz.
The difference question is also pertinent.
Of course. I regard allowing listeners to audition the difference signal as being part of the training process, whereby an artefact that might go unnoticed by a naive listener will become audible when he has learned what it sounds like. In this, I don't appear to differ from engineering orthodoxy. See, for example, http://ff123.net/training/training.html (http://ff123.net/training/training.html) , as well as the existence of the AES training CD.
John Atkinson
Editor, Stereophile
Of course. I regard allowing listeners to audition the difference signal as being part of the training process, whereby an artefact that might go unnoticed by a naive listener will become audible when he has learned what it sounds like. In this, I don't appear to differ from engineering orthodoxy. See, for example, http://ff123.net/training/training.html (http://ff123.net/training/training.html) , as well as the existence of the AES training CD.
The ff123 link you've provided does not prescribe listening to difference signals as part of the training process.
Of course. I regard allowing listeners to audition the difference signal as being part of the training process, whereby an artefact that might go unnoticed by a naive listener will become audible when he has learned what it sounds like. In this, I don't appear to differ from engineering orthodoxy. See, for example, http://ff123.net/training/training.html (http://ff123.net/training/training.html) , as well as the existence of the AES training CD.
The ff123 link you've provided does not prescribe listening to difference signals as part of the training process.
This is correct. My apologies if I was unclear. I was referring to the general topic of training listeners to identify the artefacts introduced by lossy codecs as being something with which I was in agreement.
John Atkinson
Editor, Stereophile
Single-blind still leaves a lot of room for outside influences to have an effect on the listener(s) - the whole point of ABX is to eliminate those influences so that only the possible differences in the two things being compared remain.
Despite the sarcastic remarks in my original post, I actually was unable to attend due to my work schedule. However, the description of the event on the Stereophile website, and in particular the line "Stereophile editor John Atkinson will be demonstrating the benefits of high-resolution audio and the evils of MP3" left me with the distinct impression that it would not be an evening of fact-finding, but of witch hunting. As such, even if I had been able to attend, I didn't see the point.
John, IIRC you've been doing your recording at 24/88.2 for awhile now, and I know that Stereophile has sold CDs through at least one third-party vendor, Acoustic Sounds. How about making your high-res recordings available through HD Tracks? They've got quite an impressive high-res catalog - you'd certainly be in good company. I'd buy at least one Cantus album as 24/88.2 FLACs, both because I do enjoy a capella choral music every now and then, and also to do my own ABX testing vs. 16/44 PCM and MP3 (and perhaps even AAC @ 96 kHz, which can technically support sample rates that high, although I've never tried to see if the QuickTime and/or Nero encoders will actually do so).
As an alternative, would you be willing to post <30 sec. FLACs of what you consider to be the most "challenging" or "revealing" passages of the tracks you used? I understand if legal issues prevent you from doing so, but I really would like to be able to try my own ABX of at least some of the material you used, and I suspect others on this forum would, as well.
Reading the old threads...a real can of worms, the way I see it unless samples are posted (either publically, or privately to a select trusted few) the exact same will be discussed again in 2013. Any step of signal processing could introduce the artefacts, such as the resampling.
Any step of signal processing could introduce the artefacts, such as the resampling.
I felt that resampling the reduced-data-rate files back to 24-bit/88.2kHz would eliminate the DAC and reconstruction filter as confusing variables. There would also then be no visible indication to listeners, such as the DAC sample-rate LED changing state, that the file format had changed. It also allowed me to splice the four examples together so that there was no clue anything had changed other than the possible change in sound quality. The listeners, BTW, were not told before the music segment started playing that they would be listening to four different formats, 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1kHz MP3. Only after the music had finished playing did I ask them for their reactions and tell them what they had been listening to.
John Atkinson
Editor, Stereophile
John, IIRC you've been doing your recording at 24/88.2 for awhile now, and I know that Stereophile has sold CDs through at least one third-party vendor, Acoustic Sounds. How about making your high-res recordings available through HD Tracks?
I have actually had an agreement with HDTracks agreed in principle for a year, but the legal obstacles on my end have proved more difficult to resolve than i had anticipated.
As an alternative, would you be willing to post <30 sec. FLACs of what you consider to be the most "challenging" or "revealing" passages of the tracks you used? I understand if legal issues prevent you from doing so, but I really would like to be able to try my own ABX of at least some of the material you used, and I suspect others on this forum would, as well.
Unfortunately, when I have allowed others access to some of my master files in the past, some have abused the privilege, so I am wary of doing so again, I am afraid. On the other hand, the example I used in Colorado and Seattle was the chorus "For Unto Us a Boy is Born," from the Dunedin Consort's recording of Handel's Messiah, which is available as a 24-bit/88.2kHz FLAC download from Linn Records for a couple of dollars. It was recommended to me by Linn's engineer Philip Hobbs a a good choice for revealing problems.
John Atkinson
Editor, Stereophile
The listeners, BTW, were not told before the music segment started playing that they would be listening to four different formats, 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1kHz MP3.
MP3 at 128kbps, I meant to say. (I waited too long to correct the omission.)
John Atkinson
Editor, Stereophile
The listeners, BTW, were not told before the music segment started playing that they would be listening to four different formats, 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1kHz MP3.
MP3 at 128kbps, I meant to say.
John Atkinson
Editor, Stereophile
From your choice of encoders/bitrates, I'd guess you already decided for yourself that AAC is superior...
The listeners, BTW, were not told before the music segment started playing that they would be listening to four different formats, 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1kHz MP3.
MP3 at 128kbps, I meant to say.
From your choice of encoders/bitrates, I'd guess you already decided for yourself that AAC is superior...
I chose the bitrates and formats because AAC at 320kbps is the highest bitrate supported by the iTunes/Quicktime engine and MP3 at 128kbps is typical commercial download practice. But presumably it would be a ToS#8 infringement if I were to comment on what I feel the audible differences were.
John Atkinson
Editor, Stereophile
But presumably it would be a ToS#8 infringement if I were to comment on what I feel the audible differences were.
Not if you supply ABX results to show why you "feel" that way.
I wasn't there either. I would be very interested in seeing Krabapple's questions answered.
Had you been there - had _anyone_ from HA been there - you would have known what happened at my presentation. I am puzzled by the fact that there exists simultaneously much curiosity about my Seattle presentation but not enough curiosity for anyone to attend. The same phenomenon occurred when I gave a similar presentation in Colorado 2 years ago. As I said, the next time I give a presentation like this, I'll give HA plenty of notice in case someone wishes to and is able to attend.
Are there reasons for you not wanting to answer the questions regarding which encoder, bitrate, etc was used to create the mp3 files?
Only that I did answer at length on HA the same questions, including those put to me by"krabapple," when I did the same demonstration in Colorado 2 years ago. See, for example, http://www.hydrogenaudio.org/forums/lofive...p/t71245-0.html (http://www.hydrogenaudio.org/forums/lofiversion/index.php/t71245-0.html) . See also http://www.hydrogenaudio.org/forums/index....72446&st=25 (http://www.hydrogenaudio.org/forums/index.php?showtopic=72446&st=25) .
There a lot of stuff to wade through there so correct me if I pulled out the wrong bit. This post (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=72446&view=findpost&p=639919) describes a sequence of resolutions from 88.2 kHz/24-bit to 128 kbit MP3. Listeners are apparently asked single-blind and en masse whether quality improves or degrades during playback. It has been previously demonstrated that 128 kbit MP3 is not transparent. This is not a difficult test for most listeners to pass and even if it is for for a particular listener, they have a bunch of other listeners and (since it is single blind) possibly the administer of the test to coach them to the correct answer.
The listeners, BTW, were not told before the music segment started playing that they would be listening to four different formats, 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1kHz MP3.
MP3 at 128kbps, I meant to say.
From your choice of encoders/bitrates, I'd guess you already decided for yourself that AAC is superior...
I chose the bitrates and formats because AAC at 320kbps is the highest bitrate supported by the iTunes/Quicktime engine and MP3 at 128kbps is typical commercial download practice. But presumably it would be a ToS#8 infringement if I were to comment on what I feel the audible differences were.
John Atkinson
Editor, Stereophile
The very same could be said for AAC, although iTunes Plus uses 256kbit. Still, I don't really understand what you are trying to prove with these selections. If you want to prove ANYTHING by the choice of codec, for the matter of AAC and MP3, you should definitely provide samples at the same bitrate.
Using your argument also perfectly fits mp3 - I don't know of an "mp3 engine" that is not able to decode 320kbit mp3.
On the other hand, the example I used in Colorado and Seattle was the chorus "For Unto Us a Boy is Born," from the Dunedin Consort's recording of Handel's Messiah, which is available as a 24-bit/88.2kHz FLAC download from Linn Records for a couple of dollars.
Please provide the time within this specific track (relative to the beginning of the track in mm:ss) where the difference between lossless and lossy is most obvious so that someone can provide a 30 second clip.
The listeners, BTW, were not told before the music segment started playing that they would be listening to four different formats, 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1kHz MP3.
MP3 at 128kbps, I meant to say. (I waited too long to correct the omission.)
John Atkinson
Editor, Stereophile
But were they apprised in
any way, shape or form, e.g., via the title of the presentation, the advertising or promotion for it, or anything said by way of as introduction, that your appearance would be at least partially about the sound of audio formats?
And did you continue to use Audition 1.0's not-at-all-current mp3 codec?
And did you again only solicit and note responses, such as they were, after the final (128 kbps) segment?
And if anyone in the audience said something like , 'wow, I really heard a dropoff in quality', what if any explanation did you offer?
Was the role of expectation bias in audio quality assessment mentioned at all?
The listeners, BTW, were not told before the music segment started playing that they would be listening to four different formats, 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1kHz MP3.
MP3 at 128kbps, I meant to say.
From your choice of encoders/bitrates, I'd guess you already decided for yourself that AAC is superior...
I chose the bitrates and formats because AAC at 320kbps is the highest bitrate supported by the iTunes/Quicktime engine and MP3 at 128kbps is typical commercial download practice. But presumably it would be a ToS#8 infringement if I were to comment on what I feel the audible differences were.
John Atkinson
Editor, Stereophile
I question whether 128 kbps is still typical for commercial download; certainly iTunes moved on from it as default long ago. Amazon's default is 256 kbps VBR.
And including a difference demo as a form of 'training' is just wrong for all the reasons noted in the Massenburg thread. And under those circumstances failing to explain how psychoacoustics-based lossy encoding works and what it is SUPPOSED to do, would be misleading at the very least and dishonest at worst.
On the other hand, the example I used in Colorado and Seattle was the chorus "For Unto Us a Boy is Born," from the Dunedin Consort's recording of Handel's Messiah, which is available as a 24-bit/88.2kHz FLAC download from Linn Records for a couple of dollars.
Please provide the time within this specific track (relative to the beginning of the track in mm:ss) where the difference between lossless and lossy is most obvious so that someone can provide a 30 second clip.
This download is unadulterated 24-bit/88.2kHz, of course. Following the instrumental introduction, the refrain repeats 3 times. For the version I used in my demonstration, each time the sopranos or tenors begin the verse "For unto us a boy is born" after the first -- at 1:22, 1:57, 2:38 -- I spliced seamlessly to the next version encoded with a different codec.
I am not sure how I can answer your question without conflicting with ToS#8, as I haven't tested my opinion with ABX. However, for what it's worth, I felt the sections where the choir sing "and the government shall be upon his shoulders" -- starting at 55:00, 1:33, 2:12, 2:53 -- were the most vulnerable.
John Atkinson
Editor, Stereophile
For what is it worth 128kbps is no longer the offered download bitrate, a few recent Amazon tracks were Lame -v0 (aroud 256kbps) some of the others tend to use 320kbps CBR, for practically everyone these bitrates are transparent (unless specific problem tracks are used).
The days of having a mp3 player which could only store 2 albums at 128kbps have passed 10 years ago, now iPods can store whole libraries at 320kbps.
I felt the sections where the choir sing "and the government shall be upon his shoulders" -- starting at 55:00, 1:33, 2:12, 2:53 -- were the most vulnerable.
So a 30 second clip centered at any of these times (assuming you meant 0:55 for the first one) should be acceptable?
Would you be willing to perform an ABX test and present your results as an effort of good faith?
Downloaded and standing by...
Edit: OK, I guess our Saturday morning coffee is kicking in - hot topic, indeed!
Here is the link to a 24 second section from 1:21-1:45 (the most I could fit into 8 MB, the upload limit), which encompasses both one of the sections John used in his presentation at Definitive, as well as one of the sections he feels is most revealing:
http://www.hydrogenaudio.org/forums/index....showtopic=86738 (http://www.hydrogenaudio.org/forums/index.php?showtopic=86738)
I felt the sections where the choir sing "and the government shall be upon his shoulders" -- starting at 55:00, 1:33, 2:12, 2:53 -- were the most vulnerable.
So a 30 second clip centered at any of these times (assuming you meant 0:55 for the first one) should be acceptable?
I imagine so. And yes, 0:55. Apologies for the error.
Would you be willing to perform an ABX test and present your results as an effort of good faith?
Why would that be necessary? As I said, the answer to your question implicitly involved a ToS#8 violation. I believe I showed sufficient good faith in offering an answer, in order that others might try this musical passage for themselves.
John Atkinson
Editor, Stereophile
The listeners, BTW, were not told before the music segment started playing that they would be listening to four different formats, 24-bit/88.2kHz, 16-bit/44.1kHz, 44.1kHz AAC at 320kbps, 44.1kHz MP3.
MP3 at 128kbps, I meant to say. (I waited too long to correct the omission.)
But were they apprised in any way, shape or form, e.g., via the title of the presentation, the advertising or promotion for it, or anything said by way of as introduction, that your appearance would be at least partially about the sound of audio formats?
Other than the text included in the title of this thread, no. All the attendees were told was that they were going to be listening to some of my hi-rez recordings.
And did you continue to use Audition 1.0's not-at-all-current mp3 codec?
No.
And did you again only solicit and note responses, such as they were, after the final (128 kbps) segment?
Yes.
And if anyone in the audience said something like , 'wow, I really heard a dropoff in quality', what if any explanation did you offer?
I explained what they had been listening to. After the event, not before.
Was the role of expectation bias in audio quality assessment mentioned at all?
As a general point, how can there be any expectation bias if the listeners are ignorant of what they will be listening to? And in this case, as all that people knew was that would be listening to a 24-bit 88.2kHz audio file, if there were any expectation bias, surely it would be in the opposite direction, ie, it would tend to make them _less_ critical of the lossy codecs?
John Atkinson
Editor, Stereophile
The good faith would be in your finally taking part in what is an acceptable method of testing lossy audio codecs as deemed by those who actually develop them.
And who could object to an event that promotes the idea of listening to recordings with as high a quality as possible?
Does one have to leave the comfort of one's own home to listen to such a thing?
They do if they wish to hear the hi-rez master files of my own commercial recordings...
So then it is your assertion that all other forms of those files sound degraded, or at least different?
The good faith would be in your finally taking part in what is an acceptable method of testing lossy audio codecs as deemed by those who actually develop them.
I think we should be a little more broad-minded than that. It is possible that the developers of lossy codecs have overlooked some signficiant new technology for doing sound quality evaluations. Thing is, any such technology would have to be significantly different from the methodologies that we already know, and some of us know very well.
Seeing no such thing... ;-)
I will happy await comments from members such as robert, Gabriel, menno, muaddib, Monty, r2d, Seed, C.R.Helmrich and Nick.C about the methods they use. One should not underestimate the impact HA has had on the development of lossy codecs.
Well, I just did some back-n-forth listening tests in foobar2000 between the 24 sec. FLAC sample and two 44.1 kHz LAME V2 (VBR @ ~190 kb/s) MP3s, one from a 24-bit WAV, one from a 16-bit WAV...didn't get to ABX-ing because I couldn't hear anything to ABX. I used my Sony MDR-V6 headphones being driven by the onboard RealTek HD audio of my PC (using udial and loop-back recording, I determined that setting fb2k's resampler to 88.2 kHz and its bit depth to 24 bits gave the most accurate output, and the noise floor of the headphone out is inaudible).
I downconverted to 24/44.1 and 16/44.1 WAVs using fb2k's resampler in "ultra" mode (and added dither for the 16-bit version) and then encoded those to VBR V2 MP3s using LAME (yes, I could've done that in one step but I wanted to have the WAVs in case I heard something suspect in the MP3s).
FWIW, the overall bitrate of the MP3 from the 24-bit WAV was 193 kb/s, and from the dithered 16-bit WAV it was 190 kb/s.
And who could object to an event that promotes the idea of listening to recordings with as high a quality as possible?
Does one have to leave the comfort of one's own home to listen to such a thing?
They do if they wish to hear the hi-rez master files of my own commercial recordings...
So then it is your assertion that all other forms of those files sound degraded, or at least different?
With respect, Mr. Krueger, any answer to your question would necessarily involve a ToS#8 violation. All I can do is refer you to my writings in Stereophile on this subject.
John Atkinson
Editor, Stereophile
With respect, Mr. Krueger, any answer to your question would necessarily involve a ToS#8 violation.
By simple inference from this sentence: John Atkinson admits that anything he can claim in favor of hi-rez audio is
necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects.
I can live with that and he can continue to sell his magazines (and advertising space).
With respect, Mr. Krueger, any answer to your question would necessarily involve a ToS#8 violation.
By simple inference from this sentence: John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects.
Not exactly, that only means he can't ABX the differences.
, any answer to your question would necessarily involve a ToS#8 violation.
By simple inference from this sentence: John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects.
Not exactly, that only means he can't ABX the differences.
No. It means he can't find anyone who can perceive the difference in a valid* double blind test (abx or not), or find other solid evidence of perceivable difference.
*"valid" implying things like enough trials to eliminate chance wins, randomized order, etc.
With respect, Mr. Krueger, any answer to your question would necessarily involve a ToS#8 violation.
By simple inference from this sentence: John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects.
Not exactly, that only means he can't ABX the differences.
Tos#8 is very clear on the answer Mr. Krueger requested: "8. All members that put forth a statement concerning subjective sound quality, must -- to the best of their ability -- provide objective support for their claims. Acceptable means of support are double blind listening tests (ABX or ABC/HR) demonstrating that the member can discern a difference perceptually, together with a test sample to allow others to reproduce their findings."
As my opinions and findings are based on sighted experience and single-blind tests, not ABX or ABX-HR, for me to offer any such opinion would inherently violate ToS#8. People should not draw any more from my answer than that. and please note that such statements as "John Atkinson admits..." are unsupported and subjective conjecture on the part of the person making that statement.
John Atkinson
Editor, Stereophile
No. It means he can't find anyone who can perceive the difference in a valid* double blind test (abx or not), or find other solid evidence of perceivable difference.
*"valid" implying things like enough trials to eliminate chance wins, randomized order, etc.
I doubt very much that that is true.
Downconversions to 44.1 kHz have been demonstrated to be audible in some cases, both in an article published in JAES and in an ABX test that was reported by someone here. That doesn't mean that all such downconversions will be audible, it's possible that results depend on the anti-aliasing and reconstruction filters and other aspects of the implementation, but it does mean that it's possible to set up an ABX tests which demonstrate the audibility of 44.1 kHz with some algorithms and some material.
16 bits is known, on the basis of theory, measurement, and existing psychometric data, to be inadequate to transparently reproduce the full dynamic range of acoustical audio unless noise shaping is used. See Fielder, "Dynamic-Range Issues in the Modern Digital Audio Environment," http://www.zainea.com/Dynamic%20range.htm (http://www.zainea.com/Dynamic%20range.htm). So again, it's possible to demonstrate the difference in a carefully-conducted ABX test.
There is indeed controversy here, but as in the case of 320 kbit/sec MP-3, there doesn't seem to be much doubt that one can ABX differences under certain circumstances. To me, a more interesting question would be whether the demonstrated audibility of a 44.1 kHz sampling rate is a consequence of a limit imposed by the characteristics of practical FIR filters, or whether its merely a consequence of the specific filters used in the experiments, or experimental error (rounding error, etc.).
No. It means he can't find anyone who can perceive the difference in a valid* double blind test (abx or not), or find other solid evidence of perceivable difference.
*"valid" implying things like enough trials to eliminate chance wins, randomized order, etc.
I doubt very much that that is true.
Downconversions to 44.1 kHz have been demonstrated ......
Remember that these are all in the context of Arnie's question and the response that "any answer would would necessarily involve a ToS#8 violation." and what that implies.
to be audible in some cases, both in an article published in JAES and in an ABX test that was reported by someone here. That doesn't mean that all such downconversions will be audible, it's possible that results depend on the anti-aliasing and reconstruction filters and other aspects of the implementation, but it does mean that it's possible to set up an ABX tests which demonstrate the audibility of 44.1 kHz with some algorithms and some material.
I would imagine if John was trying to show the inherently better sound of hi res recording that he wouldn't say the 16/44 sounds worse due to a bad conversion.
Tos#8 is very clear on the answer Mr. Krueger requested: "8. All members that put forth a statement concerning subjective sound quality, must -- to the best of their ability -- provide objective support for their claims. Acceptable means of support are double blind listening tests (ABX or ABC/HR) demonstrating that the member can discern a difference perceptually, together with a test sample to allow others to reproduce their findings."
As my opinions and findings are based on sighted experience and single-blind tests, not ABX or ABX-HR, for me to offer any such opinion would inherently violate ToS#8. People should not draw any more from my answer than that. and please note that such statements as "John Atkinson admits..." are unsupported and subjective conjecture on the part of the person making that statement.
What's to the best of a Stereophile editor's ability? You certainly do not lack the resources in your position. A simple double blind test should be far below the limits of your ability or resources at disposal. As such, one can only conclude that the stated inability to present objective data is self-chosen.
Now, why would an intelligent individual, both in quest of the best possible sound
and a customer base in desperate need of owning superiority-representing items, intentionally avoid placebo-eliminating protocol? Is this even worth debating about?
Is this even worth debating about?
I'd say "no". Why do we care what John Atkinson can or can't hear?
Remember that these are all in the context of Arnie's question and the response that "any answer would would necessarily involve a ToS#8 violation." and what that implies.
It may seem a quibble, but from my perspective the difference is between "can't find anyone who can perceive the difference in a valid double blind test" and "hasn't personally conducted a double blind test." And I think that's not insignificant, because in the first interpretation, we have someone who's been caught with his pants down, whereas in the second we merely have someone who can't comment on his personal experience owing to the TOS but isn't trying to hide anything because there's nothing to hide.
I would imagine if John was trying to show the inherently better sound of hi res recording that he wouldn't say the 16/44 sounds worse due to a bad conversion.
Or perhaps he'd just say what he believes? If he does have a specific belief: AFAIK, nobody really does know for sure at this point, because the objective data isn't good enough. Just subjective impressions, some blind but not double blind comparisons, and a few ABX tests/journal articles with apparently contradictory results.
I think in some regards the question is becoming moot. To use a cliche, bits are cheap, and I think audio is moving in the direction of high res downloads. Still, I'd love to see more experimentation along these lines. After all, there's a lot of legacy 44.1 kHz material out there, and it would be interesting to know if some converters can reproduce it without adding audible artifacts, or even, as some vendors claim, reduce artifacts that were introduced when the recording was made (ringing and pre-ringing, group delay, ripple, etc.). (It may even be possible to reduce aliasing; communications theory should make that impossible for a random input, but music isn't random.)
Is this even worth debating about?
I'd say "no". Why do we care what John Atkinson can or can't hear?
And why care about people who care what John Atkinson thinks he hears, either. Aren't these listening events just high-end audio gang-bangs for high-end audio swingers who enjoy a little cross-dressing (in pseudo-objective clothing). I say leave people to their fetishism (http://en.wikipedia.org/wiki/Fetish)*.
C.
* Take your pick.
Now, why would an intelligent individual, both in quest of the best possible sound and a customer base in desperate need of owning superiority-representing items, intentionally avoid placebo-eliminating protocol? Is this even worth debating about?
Honestly, I can think of lots of possible reasons, ranging from the ones you mentioned to the difficulty of arranging blind tests to the belief that they aren't valid or necessary. But I can't demonstrate the validity of any of them.
Surely, a forum that is devoted to rigorous and objective measurement of sonic attributes can avoid making unverified assertions about an individual's motivations and beliefs, or his actions at an event that nobody on the forum attended?
It's possible to disagree with Stereophile's methodology and results without making unfounded assumptions about the motivations of its editor or its readers, at least one of whom (me) has no "desperate need of owing superiority-representing items."
Maybe it's just the geek in me, but I'm more interested in seeing objective answers to questions like whether 44.1 kHz audio can be transparent, whether converters differ sonically, and whether contemporary 192 kHz/24 bit recordings (or perhaps even 44.1 kHz ones) are actually indistinguishable from the live mic feed, as some recording engineers now claim.
Maybe it's just the geek in me, but I'm more interested in seeing objective answers to questions like whether 44.1 kHz audio can be transparent, whether converters differ sonically, and whether contemporary 192 kHz/24 bit recordings (or perhaps even 44.1 kHz ones) are actually indistinguishable from the live mic feed, as some recording engineers now claim.
I have my doubts that John Atkinson is doing much (if anything) to help find them.
I must agree with googlebot's latest post. Even if JA isn't enamoured by the concept or procedure of double-blind testing, it's hardly so difficult that he couldn't, by now, have done a few quick tests to support his assertions and get them pesky crusadin' objectivists off his back. Excuse the cynic in me for wondering if perhaps he has a vested interest in ignorance (read: comforting subjectivism).
With respect, Mr. Krueger, any answer to your question would necessarily involve a ToS#8 violation.
By simple inference from this sentence: John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects.
Not exactly, that only means he can't ABX the differences.
No PaJaRo, googlebot is absolutely correct; read TOS #8. If John Atkinson wishes to change the logical conclusion to his statement, he will need to amend his statement.
Maybe it's just the geek in me, but I'm more interested in seeing objective answers to questions like whether 44.1 kHz audio can be transparent, whether converters differ sonically, and whether contemporary 192 kHz/24 bit recordings (or perhaps even 44.1 kHz ones) are actually indistinguishable from the live mic feed, as some recording engineers now claim.
I have my doubts that John Atkinson is doing much (if anything) to help find them.
Maybe, but by the same token, his measurements are a treasure trove of objective engineering data. I don't know how often I've found myself referring to them, most recently today when I was trying to help someone figure out why he was unsatisfied with the way his loudspeakers imaged in his listening room.
So while I do wish personally that Stereophile as an institution made more of an effort to address some of these questions. I don't think I can criticize JA for not pulling his weight. And we have to remember that Stereophile is itself a magazine that, whatever the breadth of its coverage, has as its focus practical criticism for the benefit of audiophiles. There are other venues, such as JAES and for that matter this one, that are more closely focused on research of this kind.
I must agree with googlebot's latest post. Even if JA isn't enamoured by the concept or procedure of double-blind testing, it's hardly so difficult that he couldn't, by now, have done a few quick tests to support his assertions and get them pesky crusadin' objectivists off his back. Excuse the cynic in me for wondering if perhaps he has a vested interest in ignorance (read: comforting subjectivism).
But where are the assertions that have to be proved? As far as I can tell, every assertion he's made in this matter is backed by ABX tests that have already been discussed on this forum. This is what has me puzzled, and, judging by a post he made on a different forum, has John puzzled as well: the audibility of these phenomena has already been objectively established. He isn't alleging anything controversial.
With respect, Mr. Krueger, any answer to your question would necessarily involve a ToS#8 violation.
By simple inference from this sentence: John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects.
Not exactly, that only means he can't ABX the differences.
No PaJaRo, googlebot is absolutely correct; read TOS #8. If John Atkinson wishes to change the logical conclusion to his statement, he will need to amend his statement.
I am really sorry If what I said was misunderstood. I just wanted to show that googlebot's sentence was not completely true. "John Atkinson admits that
anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects."
So, not anything he claims is impossible to demonstrate, only those particular examples. He could claim, hi-res audio has higher resolution than low-res audio, thus making googlebot assumption wrong.
PS: I also don't see the point why JA is not conducting/posting some ABX results to show the differences, but I also think he was overattacked on this thread
I am really sorry If what I said was misunderstood. I just wanted to show that googlebot's sentence was not completely true. "John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects."
So, not anything he claims is impossible to demonstrate, only those particular examples. He could claim, hi-res audio has higher resolution than low-res audio, thus making googlebot assumption wrong.
If having higher resolution was by itself already "in favor" of high-rez audio, which is not the case. Higher resolution for a delivery format is a waste of space if no
audible benefit can be shown.
Honestly, I can think of lots of possible reasons, ranging from the ones you mentioned to the difficulty of arranging blind tests to the belief that they aren't valid or necessary. But I can't demonstrate the validity of any of them.
...
Maybe it's just the geek in me, but I'm more interested in seeing objective answers to questions like whether 44.1 kHz audio can be transparent, whether converters differ sonically, and whether contemporary 192 kHz/24 bit recordings (or perhaps even 44.1 kHz ones) are actually indistinguishable from the live mic feed, as some recording engineers now claim.
So proper, placebo eliminating test procedures should also be in
your best interest! Then if a magazine, that you pay and with considerable advertising cash-flow, continually rejects placebo eliminating procedure (it is really not that hard if there was a will), you should ask critically instead of becoming its apologist. Treating the matter as if it was all just opinion vs. opinion, has a taste of Fox News literacy. Why would a placebo elimination protocol, as ABX, make a test invalid vs. the same test without that protocol properly implemented?* That you even parrot these "opinions" is telling.
* Please safe my time and do not answer this by whatever you regard as "authority" but with good old arguments.
With respect, Mr. Krueger, any answer to your question would necessarily involve a ToS#8 violation.
By simple inference from this sentence: John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects.
Not exactly, that only means he can't ABX the differences.
No PaJaRo, googlebot is absolutely correct; read TOS #8. If John Atkinson wishes to change the logical conclusion to his statement, he will need to amend his statement.
I am really sorry If what I said was misunderstood. I just wanted to show that googlebot's sentence was not completely true. "John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects."
So, not anything he claims is impossible to demonstrate, only those particular examples.
Please, as in earlier postings in this this thread, go by the literal meanings of what I write, not what someone else has conjectured that I "claim" or "admit."
I have not admitted that anything is "impossible," only that I have not personally performed ABX tests on the putative differences introduced by lossy codecs, etc. As has been pointed out, there have been ABX tests performed by others that demonstrate audibility, but I interpreted ToS8 as meaning that _I_ had to have performed such tests before offering my opinions.
I also don't see the point why JA is not conducting/posting some ABX results to show the differences. . .
That discussion goes well beyond the purview of this thread, I believe.
John Atkinson
Editor, Stereophile
who could object to an event that promotes the idea of listening to recordings with as high a quality as possible?
It seems to me that quality is a multi-dimensional thing. The number of samples per second and the accuracy of each sample represent two variables that impact "quality", but I think most people would object to an event that promotes the idea of 10GHz sampling for audio. This increase in "quality" is irrelevant to human listeners, while many other neglected factors may be of great importance.
The objection here is even simpler: your sighted presentation of a sliding scale from 24/88.1 to 128kbps mp3 clearly does present something where the quality reduces (I'm sure we can find an mp3 encoder which produces an easily ABXed version at 128kbps CBR), but it implies (and pre-disposes listeners to believe - and with placebo, hear) that the quality reduces in a detectable way at each stage. This is a trick. Nothing more. Whether the quality does or does not detectably reduce at each stage is impossible to determine correctly under such circumstances. This is basic psychology.
Cheers,
David.
Downloaded and standing by...
Edit: OK, I guess our Saturday morning coffee is kicking in - hot topic, indeed!
Here is the link to a 24 second section from 1:21-1:45 (the most I could fit into 8 MB, the upload limit), which encompasses both one of the sections John used in his presentation at Definitive, as well as one of the sections he feels is most revealing:
http://www.hydrogenaudio.org/forums/index....showtopic=86738 (http://www.hydrogenaudio.org/forums/index.php?showtopic=86738)
Thank you. A lovely recording.
As usual, if it lacks anything, then IMO what it lacks are a few more channels of information to begin to recreate the sound space of the performance.
For stereo, it has good breadth and even depth (it would be interesting to know how Linn mic'd it - Linn has been doing these things well for a long time now!), but I've heard other great stereo recordings augmented with more channels (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=9311&view=findpost&p=96338), and it's a drop dead gorgeous experience - a night-and-day difference. Not the smoke+mirrors / placebo / might just be there at the limits of perception stuff we're talking about here.
I'm sad that we're still pissing around with making stereo better, or moaning about the "evils" of near-as-damn-it transparent lossy encoding. It's no wonder normal, intelligent, and even quality-conscious people don't give a damn about "high end" audio any more.
Cheers,
David.
The objection here is even simpler: your sighted presentation of a sliding scale from 24/88.1 to 128kbps mp3 clearly does present something where the quality reduces (I'm sure we can find an mp3 encoder which produces an easily ABXed version at 128kbps CBR), but it implies (and pre-disposes listeners to believe - and with placebo, hear) that the quality reduces in a detectable way at each stage. This is a trick. Nothing more. Whether the quality does or does not detectably reduce at each stage is impossible to determine correctly under such circumstances. This is basic psychology.
The test material and the way that the test is conducted (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86649&view=findpost&p=743386) both contribute to the effectiveness of this. Audiences at a magic show know that what they're seeing is an illusion. As far as I know, there is no such understanding with your demonstration.
I'm sad that we're still pissing around with making stereo better, or moaning about the "evils" of near-as-damn-it transparent lossy encoding. It's no wonder normal, intelligent, and even quality-conscious people don't give a damn about "high end" audio any more.
Word!
The objection here is even simpler: your sighted presentation of a sliding scale from 24/88.1 to 128kbps mp3 clearly does present something where the quality reduces (I'm sure we can find an mp3 encoder which produces an easily ABXed version at 128kbps CBR), but it implies (and pre-disposes listeners to believe - and with placebo, hear) that the quality reduces in a detectable way at each stage. This is a trick. . .
With respect, did you not read what I have written in prior postings? The audience were _not_ told ahead of the time that the quality of the recording would change during playback. All I told them was that I was going to play them a 24-bit/88.2kHz recording by Linn's Philip Hobbs, so there was no placebo effect and if there was any expectation bias, it would work against what I wanted to demonstrate. It was only _after_ the playback that I asked them what they had noticed.
See my write-up on Philip Hobbs' identical presentation at a 2007 AES Conference: http://www.stereophile.com/content/watching-detectives (http://www.stereophile.com/content/watching-detectives) .
John Atkinson
Editor, Stereophile
With respect, Mr. Krueger, any answer to your question would necessarily involve a ToS#8 violation.
By simple inference from this sentence: John Atkinson admits that anything he can claim in favor of hi-rez audio is necessarily impossible to backup by objective evidence or scientifically established practice to exclude placebo side-effects.
Not exactly, that only means he can't ABX the differences.
If you read \TOS 8, you'll see that JA's problem is a lot broader then just ABX:
"8. All members that put forth a statement concerning subjective sound quality, must -- to the best of their ability -- provide objective support for their claims. Acceptable means of support are double blind listening tests (ABX or ABC/HR) demonstrating that the member can discern a difference perceptually, together with a test sample to allow others to reproduce their findings. Graphs, non-blind listening tests, waveform difference comparisons, and so on, are not acceptable means of providing support."
I suspect that the moderators might even accept evidence that was gathered by other means provideing that those means were at least as free from bias as the specific listening test methods that were listed. Certainly the AES has set the precident for doing this when the JAES review board approved publishing the results of the Benjamin and Gannon Jitter tests back in 1998.
Apologies - I thought I had read it all (even that previous 31 page thread), but apparently read over this.
See my write-up on Philip Hobbs' identical presentation at a 2007 AES Conference: http://www.stereophile.com/content/watching-detectives (http://www.stereophile.com/content/watching-detectives)
A nice write up. Which makes it even more bizarre that almost no one successfully ABXes this stuff.
I do like the sound (no pun intended) of an unfiltered 78rpm shellac disc as a test signal. A continuous stream of wide bandwidth impulses superimposed over audibly band limited musical content, with the former generally sitting in the middle of the "stereo" sound stage, and the latter (if tracked with a stereo cartridge) flung to all extremes of the sound stage. It kills much lossy coding. It visibly makes anti-alias filters ring like mad. But I can't hear any problems with the latter.
Cheers,
David.
Is this even worth debating about?
I'd say "no". Why do we care what John Atkinson can or can't hear?
I don't care what John can't hear, but I do care about what he *can* hear perchance that advances our understanding of the capabilities of the human ear.
IMO John throws a gigantic monkey wrench into our presumably shared search for true audio excellence by relying on only the most flawed and superficial kinds of listening tests that are known to us.
Is this even worth debating about?
I'd say "no". Why do we care what John Atkinson can or can't hear?
IMO John throws a gigantic monkey wrench into our presumably shared search for true audio excellence .......
Well, I'm not searching for true audio excellence. I'm quite happy with what I've got.
See my write-up on Philip Hobbs' identical presentation at a 2007 AES Conference: http://www.stereophile.com/content/watching-detectives (http://www.stereophile.com/content/watching-detectives) .
This report says that the lowest resolution material was 192 kbit MP3 and then later mentions 128 kbit. Your recent demonstrations are apparently 128 kbit. Is that first number an error or one-off experiment or have you done this demonstration successfully and consistently with 192 kbit MP3?
Because of the way the experiment/demonstration is run, the best we can hope to get out of it is a conclusion that the lowest resolution sample played is not transparent. If you're using a format/codec there that has already been demonstrated to be non-transparent, this result is, at best, uninteresting.
Regardless of how gracefully it is reported, it would be reckless to accept anyone's post-listening claim that they started hearing degradation after the first sample.
So, not anything he claims is impossible to demonstrate, only those particular examples. He could claim, hi-res audio has higher resolution than low-res audio, thus making googlebot assumption wrong.
Let's keep with the facts at hand please; you're moving the goal posts.
So then it is your assertion that all other forms of those files sound degraded, or at least different?
any answer to your question would necessarily involve a ToS#8 violation.
googlebot's response was the logical conclusion to the claim; there is nothing wrong or untrue about it.
I don't care what John can't hear, but I do care about what he *can* hear perchance that advances our understanding of the capabilities of the human ear.
IMO John throws a gigantic monkey wrench into our presumably shared search for true audio excellence by relying on only the most flawed and superficial kinds of listening tests that are known to us.
The audiophile is a challenging creature to study; They're elusive and highly intelligent. The endeavor feels something like a Roadrunner cartoon.
That discussion goes well beyond the purview of this thread, I believe.
Your evasion is quite telling, however; at least to me.
Scientific method requires repeatability. Since you let the cat out of the bag at the end, there is no hope for your experiment. Furthermore asking people if they heard degradation after it was done does not ensure objectivity. How many people will say they did because they were concerned about what others thought? How many people believe that they heard differences after the fact because it was told to them that differences were there?
That discussion goes well beyond the purview of this thread, I believe.
Your evasion is quite telling, however; at least to me.
I have no desire to get into a debate on the merits or lack thereof of forced-choice ABX testing. All that I think needs to be said is that I acknowledge the subject is a core value of this forum.
Scientific method requires repeatability. Since you let the cat out of the bag at the end, there is no hope for your experiment. Furthermore asking people if they heard degradation after it was done does not ensure objectivity. How many people will say they did because they were concerned about what others thought? How many people believe that they heard differences after the fact because it was told to them that differences were there?
I have no idea. As I didn't tally the results, your point is moot.
John Atkinson
Editor, Stereophile
I have no idea. As I didn't tally the results, your point is moot.
Then I guess your involvement in this discussion is done. I need only go back to my very first post criticizing your deeply flawed demonstration.
I must agree with googlebot's latest post. Even if JA isn't enamoured by the concept or procedure of double-blind testing, it's hardly so difficult that he couldn't, by now, have done a few quick tests to support his assertions and get them pesky crusadin' objectivists off his back. Excuse the cynic in me for wondering if perhaps he has a vested interest in ignorance (read: comforting subjectivism).
My recollection is that JA has in the past used his perceptions of the difficulty of doing ABX tests as an excuse not to do them. At the time he inferred that only people with deep pockets like AT&T labs and Harman International could afford to to them.
Of course what about those *megabuck*audiophiles from Michigan who call themselves SMWTMS and originated ABX tests? We must all be trust fund babies! LOL!
I like the Orwellian "forced choice" description. I think there's a political party in the US who could use JA's "talents".
Apologies - I thought I had read it all (even that previous 31 page thread), but apparently read over this.
Thanks.
See my write-up on Philip Hobbs' identical presentation at a 2007 AES Conference: http://www.stereophile.com/content/watching-detectives (http://www.stereophile.com/content/watching-detectives)
A nice write up. Which makes it even more bizarre that almost no one successfully ABXes this stuff.
This has been the subject of much discussion on a private list to which I subscribe, which includes JJ, Malcolm Hawksford, Bruno Putzeys, Peter Craven, Bob Katz, Vicky Melchior and other engineers. If there are real benefits to recording at sample rates greater than 48kHz, then why is it difficult to design an ABX test that readily reveals those benefits? One answer to that question, of course, is that there are no benefits, as was recently suggested by a poster to this thread. But personally I feel that it cannot be automatically assumed that absence of evidence is equivalent to evidence of absence.
I do like the sound (no pun intended) of an unfiltered 78rpm shellac disc as a test signal. A continuous stream of wide bandwidth impulses superimposed over audibly band limited musical content, with the former generally sitting in the middle of the "stereo" sound stage, and the latter (if tracked with a stereo cartridge) flung to all extremes of the sound stage. It kills much lossy coding. It visibly makes anti-alias filters ring like mad. But I can't hear any problems with the latter.
I have ABX'd the differences between the ringing of different low-pass filters. This was in connection with an article by Keith Howard that I published 5 years ago on the subject of reconstruction filters: http://www.stereophile.com/reference/106ringing/index.html (http://www.stereophile.com/reference/106ringing/index.html) . Keith sent me a DVD-A with unidentified music samples on it. _Very_ hard to hear any difference blind, if at all, though one thing I did note was that, after the test was over and the examples had been identified, the maximum-phase filter, where all the ringing occurs _before_ the transient, did appear to be audible under double-blind conditions. My apologies for not recalling my score.
John Atkinson
Editor, Stereophile
Because of the way the experiment/demonstration is run, the best we can hope to get out of it is a conclusion that the lowest resolution sample played is not transparent. If you're using a format/codec there that has already been demonstrated to be non-transparent, this result is, at best, uninteresting.
If the people present had not heard mp3 artifacts before and did at the demo, then I'd guess it was interesting to them.
Proving their experience as valid to anyone else might be uninteresting
If they knowingly acknowledged differences due to peer pressure, they might adopt an defensive attitude to definitive testing out of fear of not measuring up.
I have no idea. As I didn't tally the results, your point is moot.
Then I guess your involvement in this discussion is done. I need only go back to my very first post criticizing your deeply flawed demonstration.
With respect, no-one has yet offered any reason why it must have been "deeply flawed." I have addressed the substantive issues that were raised by others, and explained why they weren't relevant. I went back to your first two posts to this thread (which, please note, I did not start) and this is what you wrote:
From Message #16
I object to any presentation of lossy encoding that doesn't properly describe the mechanism by which it works and demonstrate how one goes about properly determining whether it works or does not work.
So far you have really only demonstrated to the objective-minded audio community that you are willfully ignorant of lossy encoding, unwilling to improve upon what is a clear deficiency in your presentation of the subject matter as pointed out by people who clearly know more about the subject than you do, and that you are knowingly misleading people. Personally, I find this quite shameful if not morally reprehensible.
Unless you plan on changing your behavior, John Atkinson, offering apologies for being offensive is nothing short of insincere.
From Message #19
30 minutes is more than enough time to present lossy encoding to the layman audio enthusiast in an honest way. Perhaps it's not enough time if you also want to satisfy your agenda convincingly which still appears to be at odds with an honest presentation.
I see strongly expressed opinion and objections, but no actual criticism.
John Atkinson
Editor, Stereophile
Ok, if I have been so unfair in assuming fault in your demonstration, feel free to tell us how it is worthy of our* praise.
(*) my use of our is intentional. While I do not pretend to speak for other members of this forum, it seems clear to you, at the very least, our core principle about how one is to demonstrate differences in sound quality, which is the very purpose of your demonstration.
Let's be crystal clear that the topic of discussion, as put forth by the original poster, is that you had planned a presentation demonstrating the "evils of mp3" and this presentation was likely devoid of an effort to ensure objectivity. I really should have It really said ABX, but I thought I'd reword it more broadly in order to give you the benefit of the doubt. As such, don't be surprised to experience some push-back as you continue pretend that you are helping people see the "truth". That said, I do appreciate that your recent response to David, not withstanding the desperate presentation of the Pink Elephant Orbiting Uranus argument.
Keith sent me a DVD-A with unidentified music samples on it. _Very_ hard to hear any difference blind, if at all, though one thing I did note was that, after the test was over and the examples had been identified, the maximum-phase filter, where all the ringing occurs _before_ the transient, did appear to be audible under double-blind conditions.
Well, congratulations for being able to identify the type that's nobody using exactly because of that.
<snip>
I do appreciate that your recent response to David, not withstanding the desperate presentation of the Pink Elephant Orbiting Uranus argument.
Thank you (though I thought it was the Flying Spaghetti Monster argument). It strikes me that the ringing of the different low-pass filters might be an interesting project for HA members. I still have the DVD-A with the filter examples. If Keith Howard doesn't mind, I could mail it to someone who could then rip the files and make them available to HA members so that they could ABX each type of filter against the original with Foobar. But I am not sure what would be the best way to send the key. (The tracks on the disc are not identified, of course.)
John Atkinson
Editor, Stereophile
Yay, let's ABX maximum phase filters, that have absolutely no place in audio (except maybe synthesizers and special effects), against those that we already know to be transparent!
Yay, let's ABX maximum phase filters, that have absolutely no place in audio (except maybe synthesizers and special effects), against those that we already know to be transparent!
I don't understand why you are being sarcastic. I thought that this would be something that would be appreciated on HA and I could publish the results as an addendum to Keith Howard's article. There are 7 different low-pass filters described in the article - http://www.stereophile.com/content/ringing...s-filter-page-2 (http://www.stereophile.com/content/ringing-false-digital-audios-ubiquitous-filter-page-2) - and used to prepare the musical examples on the DVD-A I mentioned. But if posters don't think this would be something they would interested in, then no harm, no foul.
John Atkinson
Editor, Stereophile
While I don't want to stand in the way of actual testing, in my opinion the presented issues are just resolved. The ear is more sensitive to pre- than post-ringing, so minimum-phase or intermediate-phase beat linear-phase (maximum phase usually isn't even discussed). Then there is the trade-off between filter steepness and ringing. If processing resources are scarce, a gentle roll-off might be preferable. But processing resources aren't scarce since at least a decade. With enough processing power (we talk about 0.1% vs. 0.2% of a modern CPU) I can optimize all low-pass parameters without negatively affecting any others. What's typically available to end users for free, for example the SRC in Foobar or SoX, can deliver so much headroom in every respect, that I just don't see the potential for actual distinction in real world tests.
I also just don't appreciate to see promoted, that questions like these would be in any way practically relevant to modern, high end music reproduction. Just moving the head 2 mm while listening has probably a hundred times more effect on a signal than anything mentioned in your linked article. And these gross non-linearities should be in the center of attention. What can I do to improve the fidelity at an actual listening position and how can I better reproduce an original sound stage in a room of completely different size? Certainly not by just buying a Stereophile approved pair of speakers, that supposedly sounded good in another room, and a tube- or other amp.
With respect, no-one has yet offered any reason why it must have been "deeply flawed." I have addressed the substantive issues that were raised by others, and explained why they weren't relevant.
The flaws in the degraded resolution demonstration: 1/ The assumption that because the subjects can hear a difference between 88.2 vs. 128 MP3, they can hear a difference the comparison between 88.2 and 44.1 that is presented inseparably in the same test. 2/ The test is conducted as a group and responses are not confidential. 3/ The test is single, not double-blind.
Thess flaws have been described in posts 57 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86649&view=findpost&p=743386), 93 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86649&view=findpost&p=743588) and 102 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86649&view=findpost&p=743642) and have not been substantively addressed.
I understand if for whatever reasons you do not consider these to be flaws and believe this is the best and proper way to conduct these evaluations. I'm here to tell you that science has already been down this road with medicine and other formerly subjective disciplines. If these methods were subjected to scientific peer review, it is quite clear they'd be thrown out promptly.
Yay, let's ABX maximum phase filters, that have absolutely no place in audio (except maybe synthesizers and special effects), against those that we already know to be transparent!
I don't understand why you are being sarcastic.
No, me neither. The basic reason we think 44.1kHz sampling should be transparent is because the action of the filter, at inaudible frequencies, should be inaudible. If a maximum phase filter is audible, then we have a problem with the theory. It doesn't matter whether anyone uses such filters or not - it raises the possibility of filters having a "sound" (even if flat/"perfect" within the audible band), which is enough to get reasonable people worried.
I'm sure I read that article (or one very like it) in print in the UK - probably in HiFi news. I've read the filter design section very carefully, and it seems to me that trusting an unwindowed inverse FFT to tell you the truth when you have been tweaking the coefficients is a serious problem. The FFT assumes a periodic signal - it calculates the spectrum for the signal it sees repeated forever. Hence the results can be misleading. Hence we don't really know that Keith's filters
were flat below 20kHz. This isn't an idle fear - the exact same issue has bitten me in the past.
There have been threads here on HA attempting to ABX 20kHz LPF maximum phase filters, correctly generated, and we've failed. I think a successful ABX of such a thing would be significant. But the key is that the filters have to be trusted. Not because what happens at 20kHZ should matter, but what happens
below 20kHz does matter - so when we're sure that everything below 20kHz is fine, then and only then can any audible problems be attributed to what's happening at/above 20kHz (and possibly other equipment in the chain interacting with what's up there).
Cheers,
David.
Why is it well established then, that minimum or intermediate phase filter designs are preferable over linear phase (in terms of perceived quality), when they must all sound equal to not fall into "a problem with the theory"?
Let alone maximum phase. We know how sensible the ear is to pre-ringing.
Have you got a link to where this test should have failed? I think with the right samples, for example pulse trains with steep attacks, it should be possible to distinguish maximum vs. minimum phase.
The basic reason we think 44.1kHz sampling should be transparent is because the action of the filter, at inaudible frequencies, should be inaudible. If a maximum phase filter is audible, then we have a problem with the theory.
Why must is be that black and white? We believe that 44.1 kHz can be transparent because we can design filters that make it transparent. So finding a filter, that doesn't, doesn't invalidate any theory.
I like the Orwellian "forced choice" description. I think there's a political party in the US who could use JA's "talents".
Just one?
Just in the US?
;-)
It can be argued that John's demonizing of the process of encouraging choices is counter-productive to the interests of his subscribers and advertisers. If audiophiles make no choices then they buy no equipment, right?
Yay, let's ABX maximum phase filters, that have absolutely no place in audio (except maybe synthesizers and special effects), against those that we already know to be transparent!
I don't understand why you are being sarcastic. I thought that this would be something that would be appreciated on HA and I could publish the results as an addendum to Keith Howard's article. There are 7 different low-pass filters described in the article - http://www.stereophile.com/content/ringing...s-filter-page-2 (http://www.stereophile.com/content/ringing-false-digital-audios-ubiquitous-filter-page-2) - and used to prepare the musical examples on the DVD-A I mentioned. But if posters don't think this would be something they would interested in, then no harm, no foul.
As others have pointed out, the harm and foul is the implicit claim that the filters evaluated in the "Ringing False" article were relevant to digital audio in 2006, when it was written. The methodology used and the choices made are unfortunately characteristic of Stereophile, which is to say woefully out-of-date and irrelevant to audio as being currently practiced in the real world.
Due to the unfortunate choices made by the author, the "Ringing False" article is so fundamentally flawed as to merit being called a straw man. Let's face it, when Stereophile is characterizing digital audio as being inferior to what is actually being delivered in $50 portable music players and the sub-$1 audio interface chips that come *free* on cheap PC motherboards, there is definitely something badly awry in Stereophile land.
It looks to me like a ploy for raising false concerns about digital audio in the interest of selling overpriced high end equipment with zero audible benefits or even deficit performance, floobydust nostrums, and "Analog Audio" panaceas.
Why is it well established then, that minimum or intermediate phase filter designs are preferable over linear phase (in terms of perceived quality), when they must all sound equal to not fall into "a problem with the theory"?
"well established"? I haven't seen any tests that would satisfy HA rules that show there's any audible difference. We're talking about 20kHz+ here, not in the audible range. Is that what you're referring to? I've no argument with you about the audible range.
Let alone maximum phase. We know how sensible the ear is to pre-ringing.
In the audible range, yes. What mechanism do you propose for the ear being sensitive to pre-ringing at frequency which are inaudible? (There are two or three possibilities, mostly to do with equipment, and they probably add together to explain the vanishing but very rarely ABXable differences experienced in blind tests).
Have you got a link to where this test should have failed? I think with the right samples, for example pulse trains with steep attacks, it should be possible to distinguish maximum vs. minimum phase.
We've had exactly that example on HA before - both in the audible range, and at 20kHz...
http://www.hydrogenaudio.org/forums/index....st&p=735236 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=85512&view=findpost&p=735236)
The basic reason we think 44.1kHz sampling should be transparent is because the action of the filter, at inaudible frequencies, should be inaudible. If a maximum phase filter is audible, then we have a problem with the theory.
Why must is be that black and white? We believe that 44.1 kHz can be transparent because we can design filters that make it transparent. So finding a filter, that doesn't, doesn't invalidate any theory.
What you say is true, but it's more subtle than that. There are things we think should make a difference (i.e. things within the audible band), and things that we think should not make a difference (i.e. things above the limit of most human hearing - and, FWIW, I suspect way above the limit of JA's hearing, as I believe he's over 60). Obviously we can design filters which aren't ideal within the audible band, causing an audible difference - and as you suggest, that doesn't mean that properly designed filters will cause an audible difference. But if changes above the limit of human hearing also turned out to be audible, then we would have a problem. It may be some cause an audible difference, and as far as we can tell others do not - but that's not really good enough - 1) we don't know why, and 2) we don't know specifically what would constitute a "proper" design, beyond empirical evidence.
Trying to design a "proper" filter, if things we believe to be inaudible turned out to be audible, would be like trying to design a decent lossy codec without any knowledge of masking or how the ear works. If that was where we were at in terms of psychoacoustics, most people wouldn't touch lossy coding with a barge poll.
Any way, this is all nice discussion... but either this can be ABXed, or it can't.
FWIW IMO there's no great harm in recording at a higher sample rate - the problem is when people claim night and day differences, or hail it as the saviour of audio, or claim they actually understand what's happening.
Cheers,
David.
I brought myself up to date and you are right, David. What I had forgotten was that the ringing frequency is a function of the cut-off frequency, so >20 kHz for the cases in discussion.
So, it would indeed be an interesting finding, if John Atkinson, who very likely can't hear a continuous, high volume 20 kHz tone, could suddenly hear 20 kHz, if it just faintly preceded other signals (and again not if it trailed them).
So, it would indeed be an interesting finding, if John Atkinson, who very likely can't hear a continuous, high volume 20 kHz tone, could suddenly hear 20 kHz, if it just faintly preceded other signals (and again not if it trailed them).
FYI, I am in my early 60s and I have my hearting tested fairly regularly. Although conventional testing only goes as high as 8kHz, my hearing sensivity below 8kHz falls within the region defined as "normal." I anecdotally test my HF cutoff on a weekly basis, as I test loudspeaker impedance by sweeping spot frequencies from 50kHz to 10Hz with the Audio Precision and noting when the tone become audible. (The output level is 6V, sourced from 600 ohms, so the spl is not high.)
I have discussed this possible paradox in Stereophile, in the context of Peter Craven's so-called "apodizing" reconstruction filter. This eliminates the conventional linear-phase "ringing" at the Nyquist frequency but introduces minimum-phase "ringing" at a slightly lower frequency. I can indeed hear neither of these frequencies with continuous tones, so you would think that the Craven filter would have no audible effect. Similarly, none of the Keith Howard low-pass filters mentioned earlier in this thread should have an audible effect.
John Atkinson
Editor, Stereophile
I have discussed this possible paradox in Stereophile, in the context of Peter Craven's so-called "apodizing" reconstruction filter. This eliminates the conventional linear-phase "ringing" at the Nyquist frequency but introduces minimum-phase "ringing" at a slightly lower frequency.
The funny part about these magic filters like the apodizing one is that indeed the majority that looks at these impulse graphs don´t realize that all the pre and post ringing they see before the main impulse is above the cut-off frequency and out of their hearing range, even at very low amplitude with any modern correctly working resampler.
For even bigger Noobs as me seing the Apodized impulse must be a big AHA effect, impressive, time to buy!
In reality creating such an impulse graph that Apodizing marketing uses is only possible with mathematics that result in a bent phase response (that alone was an audiophile no-no back a while), a bad frequency response and what i am not sure about, it must introduce TONS of aliasing.
All this bad mathematics together may indeed change the sound
I brought myself up to date and you are right, David. What I had forgotten was that the ringing frequency is a function of the cut-off frequency, so >20 kHz for the cases in discussion.
So, it would indeed be an interesting finding, if John Atkinson, who very likely can't hear a continuous, high volume 20 kHz tone, could suddenly hear 20 kHz, if it just faintly preceded other signals (and again not if it trailed them).
Just a friendly reminder that being able to hear an isolated tone at 20 KHz still doesn't confer the ability to hear bad things happening that high with real world music.
The usual sticking point for audibility up that high is masking by program material at lower frequencies such as the 12-15 KHz range, which are usually unaffected by differences among brick wall filters with corner frequencies in the ca. 20 KHz range.
Understanding masking is one of those things that we who get Zwicker and Fastl et al have over your average Joe the high end audiophile or rick the recording engineer.
So, it would indeed be an interesting finding, if John Atkinson, who very likely can't hear a continuous, high volume 20 kHz tone, could suddenly hear 20 kHz, if it just faintly preceded other signals (and again not if it trailed them).
FYI, I am in my early 60s and I have my hearting tested fairly regularly. Although conventional testing only goes as high as 8kHz, my hearing sensivity below 8kHz falls within the region defined as "normal."
It is well known that it is difficult or impossible to reliably infer the performance of either people or mechanisms above 16 Khz by looking at performance below 8 KHz.
I anecdotally test my HF cutoff on a weekly basis, as I test loudspeaker impedance by sweeping spot frequencies from 50kHz to 10Hz with the Audio Precision and noting when the tone become audible. (The output level is 6V, sourced from 600 ohms, so the spl is not high.)
I do this from time to time, and due to its casual nature, I decline to infer very much from it.
I have discussed this possible paradox in Stereophile,
What paradox? All I see is a bunch of speculations based on completely uncontrolled evaiations that can't in any way be reaosnably called tests. No way can you reasonbly make a paradox out of conflicting speculations.
in the context of Peter Craven's so-called "apodizing" reconstruction filter. This eliminates the conventional linear-phase "ringing" at the Nyquist frequency but introduces minimum-phase "ringing" at a slightly lower frequency.
This is false. What figures 1 and 2 of the relevant Stereophile review of the Meridian 808i.2 show is not an elimination of linear phase ringing, but rather a reapportionment and what seems to be a significant increase in total energy in the ringing of the so-called Apodizing filter.
Looks to me like typical high end engineering - a situation that some think is ugly to look at on a scope is swapped out for artifacts that seem to be far more energetic and at a slightly lower frequency. Neither of those are exactly what I'd call positive moves. Oh yes, and you get all of that extra ringing for a far higher price.
I can indeed hear neither of these frequencies with continuous tones, so you would think that the Craven filter would have no audible effect. Similarly, none of the Keith Howard low-pass filters mentioned earlier in this thread should have an audible effect.
As has been pointed out before JA uses what has been for decades the high end's *perfect* obfuscating smokescreen - he avoids engaging in reliable listening tests. If you can fool enough of the paying customers some of the time and fool yourself all of the time... ;-)
So, it would indeed be an interesting finding, if John Atkinson, who very likely can't hear a continuous, high volume 20 kHz tone, could suddenly hear 20 kHz, if it just faintly preceded other signals (and again not if it trailed them).
FYI, I am in my early 60s and I have my hearting tested fairly regularly. Although conventional testing only goes as high as 8kHz, my hearing sensivity below 8kHz falls within the region defined as "normal."
It is well known that it is difficult or impossible to reliably infer the performance of either people or mechanisms above 16 Khz by looking at performance below 8 KHz.
No-one has said it does, Mr. Krueger. I mentioned it because work by Sean Olive et al has shown that it is hearing loss below 8kHz that randomizes people's judgments on audio quality.
<snip>
in the context of Peter Craven's so-called "apodizing" reconstruction filter. This eliminates the conventional linear-phase "ringing" at the Nyquist frequency but introduces minimum-phase "ringing" at a slightly lower frequency.
This is false. What figures 1 and 2 of the relevant Stereophile review of the Meridian 808i.2 show is not an elimination of linear phase ringing, but rather a reapportionment and what seems to be a significant increase in total energy in the ringing of the so-called Apodizing filter.
The impulse response to which Mr. Krueger is referring can be found at http://www.stereophile.com/content/meridia...er-measurements (http://www.stereophile.com/content/meridian-8082808i2-signature-reference-cd-playerpreamplifier-measurements) . The linear phase filter "rings" at the Nyquist Frequency symmetrically before and after the single-sample impulse (fig.1). The "apodizing" filter has a null at the Nyquist frequency but "rings" only _after_ the impulse with a frequency that is of necessity lower than Nyquist (fig.2).
John Atkinson
Editor, Stereophile
For some it may be interesting in how these impulses do "ring" and how it looks spectral with Audacity.
You can see below, that all this post and pre ringing in the linear phase impulse is happening above 20kHz when done properly.
The Apodizing alike filter above is done with a non-linear sox setting.
(http://img718.imageshack.us/img718/4738/impulsapo.th.png) (http://img718.imageshack.us/i/impulsapo.png/)
I should add for completeness an example of how sox can behave when you allow alias and choose lower bandwith. For all graphs it must be said that all these differences are in a frequency range nobody should be able to spot differences.
(http://img12.imageshack.us/img12/1218/sox90a.th.png) (http://img12.imageshack.us/i/sox90a.png/)
In reality creating such an impulse graph that Apodizing marketing uses is only possible with mathematics that result in a bent phase response (that alone was an audiophile no-no back a while), a bad frequency response and what i am not sure about, it must introduce TONS of aliasing.
I don't think that's correct. The Meridian engineers report that it's linear phase up to about 18kHz. The phase response does bend above 18kHz to achieve their desired result. There's no reason there should be any aliasing at all - it can be a near-perfect low pass filter in the sense that it rejects everything above fs/2 strongly, whatever the phase response. Similarly the magnitude response in-band can be (and according to the measurements, is) flat, as it should be.
What figures 1 and 2 of the relevant Stereophile review of the Meridian 808i.2 show is not an elimination of linear phase ringing, but rather a reapportionment and what seems to be a significant increase in total energy in the ringing of the so-called Apodizing filter.
I'm not convinced there is an increase in total energy - for one thing, the graph scales are different - for another, you're judging visually while IIRC the examples generated in a previous HA thread
looked like they had different energy, when all were in fact the same (just different phase).
Cheers,
David.
The usual sticking point for audibility up that high is masking by program material at lower frequencies such as the 12-15 KHz range, which are usually unaffected by differences among brick wall filters with corner frequencies in the ca. 20 KHz range.
My bet is, if anyone ever really does here a difference, it's because the ringing at 20kHz makes it may down into the audible band due to "features" in the rest of their equipment.
But as several of us keep saying, no one has proven to a useful scientific standard that this is audible. That's the first thing that should be done, not the last!
Cheers,
David.
(http://www.microsoft.com/whdc/images/archive/convrg8c.GIF)
(http://www.microsoft.com/whdc/images/archive/ovrsamp1d.GIF)
The individual impulses are added together and the pre and post ringing is canceled out.
More pictures.
I know this is known to many people here, but it could be usefull for some.
(http://cnx.org/content/m19834/latest/graphics22.png)
(http://cnx.org/content/m19834/latest/graphics21.png)
The individual impulses are added together and the pre and post ringing is canceled out.
If there's signal content at/around the filter's cut-off frequency, the ringing certainly isn't cancelled out.
Cheers,
David.
What figures 1 and 2 of the relevant Stereophile review of the Meridian 808i.2 show is not an elimination of linear phase ringing, but rather a reapportionment and what seems to be a significant increase in total energy in the ringing of the so-called Apodizing filter.
I'm not convinced there is an increase in total energy - for one thing, the graph scales are different - for another, you're judging visually while IIRC the examples generated in a previous HA thread looked like they had different energy, when all were in fact the same (just different phase).
I just did a little graphics work with the web page, and find that the scale factors are identically the same, and that only the location of the origin is shifted vertically.
The initial pulses have slightly different heights, but the taller of the two is also narrower. I suspect that the energy content of the primary pulses are similar.
As far as the energy in the ringing goes, the Apodized wave rings longer with several times the amplitude.
I think we suspect that pre-ringing is somewhat more audible than post-ringing but not to the extent that several times the amplitude could be overcome.
Plus, the Apodized wave rings at an approx 10% lower frequency which is also a slight disadvantage.
I don't think that ringing this high (ca. 20 Khz) is audible with either filter, but were things shifted down by an octave or two, Apodizing does not seem to be advantageous at all.
That would be the interesting experiment, to move the filters' corner frequencies down until one or the other (hopefully not both!) becomes audible, to see which one becomes audible first at the highest frequency.
If it turns out that they both become audible at the same frequency, well that's Science! ;-)
My bet is, if anyone ever really does here a difference, it's because the ringing at 20kHz makes it may down into the audible band due to "features" in the rest of their equipment.
As long as the equipment is linear, ringing at an ultrasonic corner frequency is not going to come down into the audible band.
Modern equipment now has this strong tendency to be very linear (better than 0.05 % nonlinearity), even at high frequencies.
But as several of us keep saying, no one has proven to a useful scientific standard that this is audible. That's the first thing that should be done, not the last!
I recommend testing the audibility of these efffects by shifting their frequency down utnil people start hearing them.
As long as the equipment is linear, ringing at an ultrasonic corner frequency is not going to come down into the audible band.
If the equipment is linear, and human ears don't work above 20kHz, we don't need a filter
at all.
Modern equipment now has this strong tendency to be very linear (better than 0.05 % nonlinearity), even at high frequencies.
Even speakers? 0.05% = 66dB down. I'm not convinced. Though I'm doubting I could hear it even 40dB down.
I recommend testing the audibility of these efffects by shifting their frequency down utnil people start hearing them.
I can see the logic in that - you can test anti-alias filters at lower frequencies, test dither noise at higher amplitude etc - but really, if the only way to make them audible is to change them fundamentally, then they don't matter. Whereas
if they do matter - i.e. if they do cause some ABXable difference at the usual frequency or amplitude (I'm posing this is a logic problem, not fact!) then changing the frequency or amplitude to make them more audible might not be helpful at all - it might not be probing the mechanism by which they because audibly in normal use.
But it's still an interesting question: there's a point where a filter (vs no filter) becomes audible - do the differences between different filters become audible at the same frequency, or lower, or higher. It seems obvious to me that it's highly content and listener dependent - young listeners auditioning content that's rich in high frequencies and impulses should stand the best chance.
Cheers,
David.
FYI, I am in my early 60s and I have my hearting tested fairly regularly. Although conventional testing only goes as high as 8kHz, my hearing sensivity below 8kHz falls within the region defined as "normal." I anecdotally test my HF cutoff on a weekly basis, as I test loudspeaker impedance by sweeping spot frequencies from 50kHz to 10Hz with the Audio Precision and noting when the tone become audible. (The output level is 6V, sourced from 600 ohms, so the spl is not high.)
Meant to add that under these conditions, my HF cutoff is around 15kHz. Didn't mean to make it look as if I was evading the question.
John Atkinson
Editor, Stereophile
The individual impulses are added together and the pre and post ringing is canceled out.
If there's signal content at/around the filter's cut-off frequency, the ringing certainly isn't cancelled out.
Cheers,
David.
Can you explain that further?
As long as the equipment is linear, ringing at an ultrasonic corner frequency is not going to come down into the audible band.
If the equipment is linear, and human ears don't work above 20kHz, we don't need a filter at all.
Modern equipment now has this strong tendency to be very linear (better than 0.05 % nonlinearity), even at high frequencies.
Even speakers? 0.05% = 66dB down. I'm not convinced. Though I'm doubting I could hear it even 40dB down.
I was thinking about electronics. But, about those loudspeakers...
We're talking really high frequencies. Tweeters with better than 0.1 % THD at high frequencies are not rare. I've owned a number that performed that well on the test bench.
Loudspeaker nonlinearity *is* a problem at low and middle frequencies. Once you get well above the highest crossover frequency, things often settle out.
Also, the amplitude of the very high frequency content in musical program material is usually way down, like 30-40 dB below midrange. and bass.
Just about everything is pretty linear if you hit it softly enough. ;-)
I recommend testing the audibility of these efffects by shifting their frequency down utnil people start hearing them.
I can see the logic in that - you can test anti-alias filters at lower frequencies, test dither noise at higher amplitude etc - but really, if the only way to make them audible is to change them fundamentally, then they don't matter.
I agree with the idea that if going to these extremes is the only way to get positive results then they don't matter. However, I'm also a believer in basing my thinking on positive results wherever possible. So, if I know that something becomes audible only in this extreme case, then I'm a lot more comfortable saying that I know that it doesn't matter in the less extreme cases.
This also helps me predict when things become problematical. For example I know that 20 repetitions of D -> A -> D conversion with certain converters is not audible, but 40 repetitions are audible. With other converters the corresponding numbers are 5 and 10. And this converter over here can be heard in just one pass. Knowing all that I feel like I can start choosing components and configuring systems with some confidence.
Whereas if they do matter - i.e. if they do cause some ABXable difference at the usual frequency or amplitude (I'm posing this is a logic problem, not fact!) then changing the frequency or amplitude to make them more audible might not be helpful at all - it might not be probing the mechanism by which they because audibly in normal use.
That is a philosophical possibility, but I don't know of any cases where it is an observable reality. Educate me!
But it's still an interesting question: there's a point where a filter (vs no filter) becomes audible - do the differences between different filters become audible at the same frequency, or lower, or higher. It seems obvious to me that it's highly content and listener dependent - young listeners auditioning content that's rich in high frequencies and impulses should stand the best chance.
I say line up the young listeners and tough program material and let the listening tests begin! However, JJ tells me that there are some artifacts that are most noticable to people with certain kinds of hearing damage.
My own perosnal expereince tells me that there are people with hearing damage that are very intolerant of even just moderate listening levels. Hyperacoustisis or some such. Mostly older people. So, listening tests are not just for the young or those with normal hearing.
That is a philosophical possibility, but I don't know of any cases where it is an observable reality.
Me Neither. At least, not with working ears and equipment. I'm sure we can find broken ears and/or equipment where it's true - i.e. there's one detection mechanism at some such amplitude or frequency, then that detection mechanism fails as you reduce amplitude or increase frequency (=inaudible) - but then some other detection mechanism kicks in further on down or up the scale (=audible again).
silly example: if you knew nothing about digital audio, and had an undithered 10-bit ADC+DAC in a box, you'd observe horribly increasing distortion as you increased the signal level (due to clipping). You'd observe the distortion reduced to very little as you decreased the signal level (below clipping). Magic - at that level the distortion is inaudible - so obviously so, and so dramatically, that who could doubt it? However, if you reduce the signal further and further, eventually you'll find that the distortion increases again (due to undithered quantisation noise).
Ultimately it's almost the same thing: a sine wave getting turned into a square wave - but happening at wildly different levels. Due to different mechanisms. And due to not doing things properly (no dither!).
Well, if there's some funny thing in human ears like that, and the ringing is audible in two different ways via two different mechanisms, then your test finds one thing, while the effect is still audible (marginally) under very different conditions. e.g. vibration of the basilar membrane detected by working inner hair cells in the region of the vibration (i.e. audible ringing - audible sound!) becomes inaudible at some such frequency BUT vibration of the basilar membrane in a region where there are no working inner hair cells, such that this motion changes the potential of outer hair cells, is marginally detectable at pretty much any frequency.
Given the complexity of the ear, and the persistent reports of something strange happening, this may be either Russell's Teapot with a huge dose of placebo, or else something that is close to reality. I'm not sure which.
IMO It's not quite on the level of homoeopathy. i.e. there's no mechanism in the world by which water can have a memory, whereas there are plenty of mechanisms by which imperfect DACs, amps, speakers and listeners can combine to make 96kHz sampling sound different from 44.1kHz sampling. So when we get positive ABX results (there have been a very few right here), we have to figure out whether it's down to things about ears that we don't understand, or due to some equipment that's not sufficiently "perfect". And so far no one has (to my satisfaction).
Cheers,
David.
I don't think that's correct. The Meridian engineers report that it's linear phase up to about 18kHz. The phase response does bend above 18kHz to achieve their desired result. There's no reason there should be any aliasing at all - it can be a near-perfect low pass filter in the sense that it rejects everything above fs/2 strongly, whatever the phase response. Similarly the magnitude response in-band can be (and according to the measurements, is) flat, as it should be.
On the Stereophile Meridian 808.2 Signature review that Peter Craven describes their Apodizing filter as new minimum-phase filter that begins rolling off below the original Nyquist frequency.
I don't think that's correct. The Meridian engineers report that it's linear phase up to about 18kHz. The phase response does bend above 18kHz to achieve their desired result. There's no reason there should be any aliasing at all - it can be a near-perfect low pass filter in the sense that it rejects everything above fs/2 strongly, whatever the phase response. Similarly the magnitude response in-band can be (and according to the measurements, is) flat, as it should be.
On the Stereophile Meridian 808.2 Signature review that Peter Craven describes their Apodizing filter as new minimum-phase filter that begins rolling off below the original Nyquist frequency.
I studied up on the concept of *Apodizing* filters and found that in optics, an apodizing filter is a kind of a correcting filter. The example that was given would be a Gaussian filter that was either too light or too dark in the middle. A correcting or Apoizing filter would be added to obtain the desired response.
Catalog listing for optical apodizing filters (http://www.edmundoptics.com/onlinecatalog/displayproduct.cfm?productid=3121)
I conclude that an audio apodizing filter might be a reduced-ringing low pass filter that reduces energy at high frequencies that a sharp cut-off filter whose corner frequency is at a higher frequency would turn into ringing. IOW follow a linear phase filter with a 22.05 KHz corner frequency by a minimum phase filter with a 20 KHz corner frequency to attenuate the linear phase filter's ringing at 22.05 KHz. The apodizing filter need not provide a ton of attenuation, since the liner phase filter still reduces out-of-band responses.
Here is some interesting stuff why Benchmark Media doesn´t use Apodizing. They find its non-linearity already being a problem and they are prone to high levels of image fold back, what to my understanding is aliasing.
http://www.benchmarkmedia.com/discuss/foru...odizing-filters (http://www.benchmarkmedia.com/discuss/forum/general-conversation/apodizing-filters)
On the same page they even find that there is no good evidence pre-ringing is a problem at 44.1kHz at all.
Interesting how even the business differs in seing things here.
Edit: Not the first time i have read some things at Benchmark Media or related reviews that make them look like a pretty honest manufacturer of well engineered high quality gear.
Other than the text included in the title of this thread, no. All the attendees were told was that they were going to be listening to some of my hi-rez recordings.
The title of this thread is "Audiophile" listening event @ Definitive Audio in Seattle, Atkinson to demonstrate "evils of MP3"
The Stereophile blurb for the event was "John Atkinson will be demonstrating the benefits of high-resolution audio and the evils of MP3, using the master files of some of his recordings. "
So, what was the title of the presentation?
And did you continue to use Audition 1.0's not-at-all-current mp3 codec?
No.
Which codec and settings were used for the 128kbps mp3s?
And did you again only solicit and note responses, such as they were, after the final (128 kbps) segment?
Yes.
in that case, see previous question.
I explained what they had been listening to. After the event, not before.
There's more than one explanation for the results you have reported. Which one did you offer?
As a general point, how can there be any expectation bias if the listeners are ignorant of what they will be listening to? And in this case, as all that people knew was that would be listening to a 24-bit 88.2kHz audio file, if there were any expectation bias, surely it would be in the opposite direction, ie, it would tend to make them _less_ critical of the lossy codecs?
The assumes too much. Two issues outstanding are 1) whether the responders were (unwittingly or not) reporting a difference between the penultimate format and the final format , rather than between each format, and 2) whether there was any source of conscious or unconscious bias. The latter are hard to rule out without rigorous methods, as you well know.
So I wonder how assertive your 'explanations' were, and how much uncertainty you acknowledged.
It was only _after_ the playback that I asked them what they had noticed.
Which, itself, can be a 'loaded' question. Particularly when asked by an editor of an 'audiophile' journal that cites 'the evils of MP3', speaking at a gathering of audiophiles.
Please do try to bring this roadshow to NYC. Short of someone posting a video, I suspect witnessing it myself is the only way I'll get substantive answers to my questions. They require more than just one-word responses.
Why is it well established then, that minimum or intermediate phase filter designs are preferable over linear phase (in terms of perceived quality), when they must all sound equal to not fall into "a problem with the theory"?
"well established"? I haven't seen any tests that would satisfy HA rules that show there's any audible difference. We're talking about 20kHz+ here, not in the audible range. Is that what you're referring to? I've no argument with you about the audible range.
Well done JA. You've generated a nice diversion from discussion of your 'listening event'.
Mods, perhaps this discussion of filter audibility should be split off as a new topic?
Why is it well established then, that minimum or intermediate phase filter designs are preferable over linear phase (in terms of perceived quality), when they must all sound equal to not fall into "a problem with the theory"?
"well established"? I haven't seen any tests that would satisfy HA rules that show there's any audible difference. We're talking about 20kHz+ here, not in the audible range. Is that what you're referring to? I've no argument with you about the audible range.
Well done JA. You've generated a nice diversion from discussion of your 'listening event'.
It wasn't meant as a diversion, merely that as "2BeDecided" had mentioned the ringing of low-pass filters, I thought that HA posters would be interested in ABXing the files created by Keith Howard using 7 different filters that accompanied his January 2006 Stereophile article.
Regarding your multiple follow-up questions in recent posts, I feel it would be more worthwhile, as you suggested, your attending my next New York presentation (date tba) and experiencing the demonstration in person. (I do note that you asked some of the same questions on HA two years ago, and I did offer answers at that time.) In the meantime, as someone has posted a link to a segment of the original Linn recording, you can perform your own tests of the original vs various data-reduced versions.
John Atkinson
Editor, Stereophile
So proper, placebo eliminating test procedures should also be in your best interest! Then if a magazine, that you pay and with considerable advertising cash-flow, continually rejects placebo eliminating procedure (it is really not that hard if there was a will), you should ask critically instead of becoming its apologist. Treating the matter as if it was all just opinion vs. opinion, has a taste of Fox News literacy. Why would a placebo elimination protocol, as ABX, make a test invalid vs. the same test without that protocol properly implemented?* That you even parrot these "opinions" is telling.
* Please safe my time and do not answer this by whatever you regard as "authority" but with good old arguments.
I'm afraid you've lost me! I believe I already said that I personally wish that Stereophile did more blind testing, though I won't go so far as to say "it is really not that hard if there was a will," because I'm not personally familiar with the practical constraints under which the magazine operates. Common sense suggests that it would be more practical for some devices -- DAC's, say, or preamps -- than for others -- loudspeakers, say, or power amps (since for example high output impedance power amplifiers can interact with the complex impedance of a specific loudspeaker).
As to ABX tests, they have some limitations having to do with the difficult of detecting differences with a low probability of detection to a 95% CI in tests of practical scope. See Les Leventhal's letter here (second one on the page):
http://www.stereophile.com/content/highs-l...-testing-page-2 (http://www.stereophile.com/content/highs-lows-double-blind-testing-page-2)
There are those who claim other limitations as well, such as the confusion and stress of the test, but I'm not aware of any scientific evidence to back up those assertions, so I'll put them in the "possibility" category.
I'm sad that we're still pissing around with making stereo better, or moaning about the "evils" of near-as-damn-it transparent lossy encoding. It's no wonder normal, intelligent, and even quality-conscious people don't give a damn about "high end" audio any more.
Cheers,
David.
Since it's established that even high bit rate lossy encoding produces the occasional artifact, minor though they may be, I'm not sure why anyone would bother with lossy encoding at this point, except in a bandwidth-constrained medium such as radio. But I wanted to say that I agree 100% with your comment about stereo. And might add to that vinyl and other obsolete technologies. The counterargument, which someone made to me a few days ago, is that audiophiles are pretty much "stuck" with these formats because of the availability of source material.
Common sense suggests that it would be more practical for some devices -- DAC's, say, or preamps -- than for others -- loudspeakers, say, or power amps (since for example high output impedance power amplifiers can interact with the complex impedance of a specific loudspeaker).
Common sense suggests it is very practical for the topic at hand, which is not loudspeakers or power amps.
Common sense suggests it is very practical for the topic at hand, which is not loudspeakers or power amps.
But then, it's settled, insofar as lossy codecs are concerned.
OTOH, after reading some very interesting posts about preringing and apodizing filters, I'm just as flummoxed as I was by questions about the audibility of filters and whether sampling rates above 44.1 (possibly 48) kHz have any audible benefit in modern equipment. The apparently conflicting ABX tests don't help. Or the conflicting tests on the audibility threshold of jitter.
It seems like fertile ground for research, and one that doesn't require a speaker shuffler. :-) Though I'm not sure how you'd design a meaningful comparative testing regime for a magazine, if you wanted to do so, and if it was warranted, that is, if it were demonstrated that there were audible differences between players, converters, and sampling rates.
But then, it's settled, insofar as lossy codecs are concerned.
...and it is settled as far as high-resolution audio is concerned as well.
Lest we forget Josh358, the topic at hand is JA's willful avoidance of proper objective testing methods.
Lest we forget Josh358, the topic at hand is JA's willful avoidance of proper objective testing methods.
That brings me down to some boring repeatable no-evidences around the net
I find myself to often discussing "theoretical" cosmetics on how to handle audio files...
I have to praise Hydrogenaudio once more not to rely on that sillly esotheric sound experience bs
Regarding your multiple follow-up questions in recent posts, I feel it would be more worthwhile, as you suggested, your attending my next New York presentation (date tba) and experiencing the demonstration in person. (I do note that you asked some of the same questions on HA two years ago, and I did offer answers at that time.) In the meantime, as someone has posted a link to a segment of the original Linn recording, you can perform your own tests of the original vs various data-reduced versions.
John Atkinson
Editor, Stereophile
Apparently a thing or two has changed in the past two years (e.g., you say you're no longer using Audition 1.0 to generate 128 kbps mp3s...but not saying what you ARE using now.) It should be simple to answer this one at least:
what was the title of your presentation in Seattle?
As to ABX tests, they have some limitations having to do with the difficult of detecting differences with a low probability of detection to a 95% CI in tests of practical scope. See Les Leventhal's letter here (second one on the page):
http://www.stereophile.com/content/highs-l...-testing-page-2 (http://www.stereophile.com/content/highs-lows-double-blind-testing-page-2)
The stats are beyond me, but I think there were concerns raised about this article on HA at the time.
While the actual maths of the stats is beyond me, the common sense intuition of coin flipping is probably within the grasp of anyone...
Thus, with a 16-trial listening test analyzed at the conventional .05 level of significance, the probability of the investigator overlooking differences so subtle that the listener can correctly identify them only 60% of the time is a whopping .8334! Accordingly, when true differences between components are subtle, it is not surprising that 16-trial listening tests with (or without) thc ABX comparator typically fail to find them.
Think about that for a second - "differences so subtle that the listener can correctly identify them only 60% of the time" - well, on average you guess right 50% of the time anyway, so in this situation the listener is doing better than chance only 10% of the time - i.e. there's only one trial in ten when they actually
notice the difference!
A 1-in-10 chance, only checked 16 times - and each 9-in-10 "miss" delivers random data. Of course you're more likely than not to miss it if you only do 16 trials.
But what are we suggesting here? We're getting someone to do an ABX test. We're letting them choose the programme material, the listening room, how long they listen, and allowing them to switch between A, B and X as much as they want before committing to a decision. Despite all this, they only hear a difference 1 time out of 10. In only 16 trials, this fact is missed.
And somehow this 1 time out of 10 difference is supposed to be the same difference that, in a sighted trial, is immediately obvious to them the moment the equipment is switched on - a difference so significant that describing it take three pages of flowery prose in Stereophile? A difference so robust that it's not impaired by the fact they're being paid to do the review, have a copy deadline to meet, and (typically) can't easily switch between this and another piece of equipment without getting up and (at least) changing some cables/connections?
I think they're taking the Michael.
I can't find it, but I'm sure there's a thread where someone found a slight difference when encoding a Madonna track to Musepack (or maybe it was mp3) - it was near as damn it perfect, but someone (I think it was Garf) got a positive ABX result by running some insane number of trials (50, or 100, or 150 - something like that). But at that kind of level, the tester's honest description of the audible difference isn't going to read like a Stereophile review - it usually reads "I though I was just guessing, but it turned out there was something in it", or "the difference I thought I was hearing seemed to change, and I had to keep resting, but I finally got a significant result so there's something real there", or whatever.
Cheers,
David.
...I personally wish that Stereophile did more blind testing,
Given that it is arguable that they never really did any that were credible...
I won't go so far as to say "it is really not that hard if there was a will," because I'm not personally familiar with the practical constraints under which the magazine operates.
Other magazines have done it. These days people do ABX tests at home the same day they get the idea they want to don one, and for a zero out-of-pocket cost.
Common sense suggests that it would be more practical for some devices -- DAC's, say, or preamps -- than for others -- loudspeakers, say, or power amps (since for example high output impedance power amplifiers can interact with the complex impedance of a specific loudspeaker).
Frankly, testing high output impedance power amps is a waste of time because their performance is so weird and dependent on their operational environment. One of the basic requirements of a reasonable tset is that the item you are testing has properties that are somewhat stable and reasonably independent of their operational context. There's no reason to test something that changes dramatically every time you change its operational environment because your test results have zero generality.
As to ABX tests, they have some limitations having to do with the difficult of detecting differences with a low probability of detection to a 95% CI in tests of practical scope. See Les Leventhal's letter here (second one on the page):
http://www.stereophile.com/content/highs-l...-testing-page-2 (http://www.stereophile.com/content/highs-lows-double-blind-testing-page-2)
That article is very old news. Les Leventhal was dealt with on a professional basis by The Audio Engineering Society.
The basic problem is that there are no known reliable bias-controlled listening test methodologies that produce the results that magazines like Stereophile need to justify their existence and support their credibility. Many people here understand that they are basically technically useless advertising vehicles. They do contain some valid technical information, but its there to create a perception of factuality that they generally lack.
There are those who claim other limitations as well, such as the confusion and stress of the test, but I'm not aware of any scientific evidence to back up those assertions, so I'll put them in the "possibility" category.
Leventhal's suppositions are no more credible than the whining about confusion and stress. The real problem is that we have a large segment of the audio industry that is based on fallacious assertions and bad logic.
It should be simple to answer this one at least:
what was the title of your presentation in Seattle?
There was no formal title. I was introduced to the audience with the words "And now John Atkinson of Stsreophile will play some of his high-resolution recordings."
John Atkinson
Editor, Stereophile
I can't find it, but I'm sure there's a thread where someone found a slight difference when encoding a Madonna track to Musepack (or maybe it was mp3) - it was near as damn it perfect, but someone (I think it was Garf) got a positive ABX result by running some insane number of trials (50, or 100, or 150 - something like that). But at that kind of level, the tester's honest description of the audible difference isn't going to read like a Stereophile review - it usually reads "I though I was just guessing, but it turned out there was something in it", or "the difference I thought I was hearing seemed to change, and I had to keep resting, but I finally got a significant result so there's something real there", or whatever.
Cheers,
David.
What David said. For a period years ago, Les Leventhal's JAES articles were the cudgel of choice wielded by the Stereophile/TAS faithful against DBTs. The fact that the differences typically asserted by the Mikey Fremers of the world seem quite apparent to them after brief audition, and therefore aren't what Leventhal is talking about, went by the wayside.
I'm afraid you've lost me! I believe I already said that I personally wish that Stereophile did more blind testing, though I won't go so far as to say "it is really not that hard if there was a will," because I'm not personally familiar with the practical constraints under which the magazine operates.
Constraints? Well, it does sell advertising to lots of manufacturers and vendors who might not take kindly to 'no difference supported' DBT results....
But the audiophile press' antipathy to DBT seems as much philosophical -- bordering on religious -- as anything else. I have heard Mr. Atkinson tell the tale of his Damascene conversion from DBT advocate (though I'm not sure how deep that ever ran) to one who seems to find DBT quite beside the point. IIRC he set up a DBT between an amp* he liked and another amp that was cheaper. The DBT didn't support an audible difference, so he went with the cheaper amp. Some time later he found himself dissatisfied, swapped in the tube amp, and all was bliss again. So to him, that meant DBTs aren't useful.
Now to me, the thing to do would be to re-do a DBT *then*, when presumably one is sensitized to the faulty 'sound' of the 2nd amp. (Indeed, audiophiles are forever complaining that the DBTs they read about didn't allow enough time for the listener to 'learn' the sound of the devices under test. One would think a clear published demonstration of this need, by Mr. Atkinson, would be a boon to their argument.) I asked Mr. Atkinson why he didn't try that - his response, more or less, was that he didn't see the point.
(* an interesting twist here: I seem to recall that amp #1 was a tube amp, and #2 was an SS amp -- so
a priori, a positive DBT result would not be as remarkable as SS vs SS)
But then, it's settled, insofar as lossy codecs are concerned.
...and it is settled as far as high-resolution audio is concerned as well.
Lest we forget Josh358, the topic at hand is JA's willful avoidance of proper objective testing methods.
But is the audibility of high res settled? I know of several sets of ABX tests, including two published in JAES and one conducted here. The results are contradictory. So -- are the contradictions the result of experimental error? Specific equipment or algorithm limitations? Fundamental limitations on practical FIR filters, or obscure mechanisms, such as intermodulation in the loudspeakers, air, or ears?
So far, it seems to me that our knowledge belongs in the Journal of Irreproducible Results.
For me, anyway, this is more interesting than Stereophile's failure to do more objective testing, a policy with which, as I've said, I happen to disagree, or with the absence of scientific controls in an informal demonstration, something with which I have no argument, having witnessed over the years literally thousands of informal demonstrations at conventions, trade shows, and meetings, and having found some of them informative and educational.
As to ABX tests, they have some limitations having to do with the difficult of detecting differences with a low probability of detection to a 95% CI in tests of practical scope. See Les Leventhal's letter here (second one on the page):
http://www.stereophile.com/content/highs-l...-testing-page-2 (http://www.stereophile.com/content/highs-lows-double-blind-testing-page-2)
The stats are beyond me, but I think there were concerns raised about this article on HA at the time.
While the actual maths of the stats is beyond me, the common sense intuition of coin flipping is probably within the grasp of anyone...
Thus, with a 16-trial listening test analyzed at the conventional .05 level of significance, the probability of the investigator overlooking differences so subtle that the listener can correctly identify them only 60% of the time is a whopping .8334! Accordingly, when true differences between components are subtle, it is not surprising that 16-trial listening tests with (or without) thc ABX comparator typically fail to find them.
Think about that for a second - "differences so subtle that the listener can correctly identify them only 60% of the time" - well, on average you guess right 50% of the time anyway, so in this situation the listener is doing better than chance only 10% of the time - i.e. there's only one trial in ten when they actually notice the difference!
A 1-in-10 chance, only checked 16 times - and each 9-in-10 "miss" delivers random data. Of course you're more likely than not to miss it if you only do 16 trials.
But what are we suggesting here? We're getting someone to do an ABX test. We're letting them choose the programme material, the listening room, how long they listen, and allowing them to switch between A, B and X as much as they want before committing to a decision. Despite all this, they only hear a difference 1 time out of 10. In only 16 trials, this fact is missed.
And somehow this 1 time out of 10 difference is supposed to be the same difference that, in a sighted trial, is immediately obvious to them the moment the equipment is switched on - a difference so significant that describing it take three pages of flowery prose in Stereophile? A difference so robust that it's not impaired by the fact they're being paid to do the review, have a copy deadline to meet, and (typically) can't easily switch between this and another piece of equipment without getting up and (at least) changing some cables/connections?
I think they're taking the Michael.
I can't find it, but I'm sure there's a thread where someone found a slight difference when encoding a Madonna track to Musepack (or maybe it was mp3) - it was near as damn it perfect, but someone (I think it was Garf) got a positive ABX result by running some insane number of trials (50, or 100, or 150 - something like that). But at that kind of level, the tester's honest description of the audible difference isn't going to read like a Stereophile review - it usually reads "I though I was just guessing, but it turned out there was something in it", or "the difference I thought I was hearing seemed to change, and I had to keep resting, but I finally got a significant result so there's something real there", or whatever.
Cheers,
David.
Whoa, there's a lot here. So let me begin by saying that, to a large extent, I agree. I don't know how many times I've heard people say that, when they tried a blind test, the differences they thought they heard either disappeared or became much subtler. I think we've all seen flowery reviews in which minor differences (if they're real) are presented as if they were major ones, or in which lateral or even backwards moves are presented as progress.
Also, for me, one of the benefits of ABXing is that even if practical tests do overlook some differences, it tends to separate the obvious ones from differences that, if they're real, are extremely subtle. This makes blind testing a very useful tool for manufacturers who are trying to design to a price point, and some manufacturers do use it that way.
That being said, I think you've misrepresented the technique of subjective reviewers, who typically listen to a component for a relatively lengthy period, to become familiar with its idiosyncracies.
In theory, it's possible to conduct an ABX test of any length, but in practice, there are practical constraints. So if lengthy listening does in fact have benefits, they will tend to be lost in an ABXing regime.
Another argument against ABX testing is that its better suited to basic psychometric evaluations with test signals than it is to music, which is a complex signal and puts great demands on short term memory. This would also be one of the arguments for lengthy testing, since long term memory has a greater capacity than short term memory.
OK, so I've presented the arguments. The problem is, I don't know how to demonstrate, objectively, that they're correct or not, without recourse to the statistics. And the statistics can only tell us so much. Statistics can quantify the practical difficult of detecting subtle differences in ABX tests, but it can't demonstrate with certainty that such differences exist, or how common they are.
I can, however, make a recording that will fool almost any ABX test of practical length. I need only record a highly intermittent but obvious audio flaw. The computer audio I'm listening now has at least three such flaws. One is apparently caused by a problem on the motherboard audio chipset. Every few days, it emits some fairly loud chirps. Obvious to anybody, but unlikely to be ABX'd. Another is a buzzing, only on some notes, pretty much only on piano. Probably caused by flaking of the protective coating on the neodynium bar magnets in the speakers. Another would be highly frequency-dependent planar resonances, again, obviously audible only on specific cuts.
They're all examples of low probability events, which per Leventhal's analysis would require an inordinate number of trials for a 95% CI.
What I can't do is extend this to the other sort of subtlety that subjective listeners say they hear, such as differences in soundstaging, bass, grain, etc. It seems obvious that some of these differences, if they were nearly as pronounced as the listeners claim, would show up right away on an ABX comparison. But I don't know how to test whether subtler differences might or might not. The problem rapidly becomes circular, because even if you introduce a known degree of distortion, you have no objective way of knowing whether it's audible without recourse to the very ABX test you're testing.
In theory, it's possible to conduct an ABX test of any length, but in practice, there are practical constraints. So if lengthy listening does in fact have benefits, they will tend to be lost in an ABXing regime.
Nonsense.
Another argument against ABX testing is that its better suited to basic psychometric evaluations with test signals than it is to music, which is a complex signal and puts great demands on short term memory. This would also be one of the arguments for lengthy testing, since long term memory has a greater capacity than short term memory.
Again, nonsense.
it can't demonstrate with certainty that such differences exist, or how common they are.
"Flying spaghetti monster" argument duly noted.
Another argument against ABX testing is that its better suited to basic psychometric evaluations with test signals than it is to music, which is a complex signal and puts great demands on short term memory. This would also be one of the arguments for lengthy testing, since long term memory has a greater capacity than short term memory.
That's one of Bob Stuart's arguments. I don't think it stacks up...
I can, however, make a recording that will fool almost any ABX test of practical length. I need only record a highly intermittent but obvious audio flaw. The computer audio I'm listening now has at least three such flaws. One is apparently caused by a problem on the motherboard audio chipset. Every few days, it emits some fairly loud chirps. Obvious to anybody, but unlikely to be ABX'd. Another is a buzzing, only on some notes, pretty much only on piano. Probably caused by flaking of the protective coating on the neodynium bar magnets in the speakers. Another would be highly frequency-dependent planar resonances, again, obviously audible only on specific cuts.
In both critiques, it seems apparent to me that the "good" way of doing something like existing subjective testing, but with double-blind statistical certainty, is firstly to do double blind A/B testing where there's no limit on time or source material. Listen to A for a month if you want - play all the music you own (OK, that would take a few years for some of us, but you get the point). Then have a go at B. Then try direct comparisons if you wish.
Then you know what you think the differences are, you know their nature, you know what kind of content reveals them etc. Now you can go for a full ABX to prove that it's real.
They're all examples of low probability events, which per Leventhal's analysis would require an inordinate number of trials for a 95% CI.
They're not though. If a slightly broken speaker cone reveals problems with solo piano music, you run 16 ABX trial with piano music - not one each with each random CD you one. If there's a highly intermittent fault (even a subtle one) you pick X=A or X=B when you hear it - you don't pick anything until you do.
The fact that magazine rarely is ever do double blind A/B testing, never mind the X part, speaks volumes IMO.
The fact that you could actually do a full standard sighted test, and then ABX whatever you found to be most revealing - AND THEN PEOPLE USUALLY FAIL - is also quite strange. Or not.
There's no great excuse against blind A/B - certainly not where all the testing happens 9-5 in the magazine's office. Obviously at home there are other issues, but it's not insurmountable.
I don't really think you should publish reviews if you can't manager to do them properly. But then, it's a free market. There are people who want to pay to read flawed test reports.
Cheers,
David.
Given that it is arguable that they never really did any that were credible...
That I can't comment on, not having been privy to the debate.
Other magazines have done [blind testing]. These days people do ABX tests at home the same day they get the idea they want to don one, and for a zero out-of-pocket cost.
JA is really far more qualified to discuss this than I. But I think it has to be remembered that what is zero out-of-pocket for us isn't for a commercial endeavor, and that Stereophile's primary focus is on judging equipment in highly optimized settings. Frankly, it's not a strategy with which I entirely agree, because I believe that choosing equipment with (alleged) sonic characteristics that counterbalance the sonic characteristics of other equipment leads to impractical complications. But I have seen the consequences of tests in which I believe insufficient attention was paid to interface and setup, as for example Harman's test of a Martin Logan hybrid in mono, in acoustics and at a distance that may not have been appropriate for that speaker.
But while, as I've said, I do wish Stereophile would ABX some components, in particular the more controversial ones such as esoteric cables and power cords, and I'd like to see blind testing done whenever that is practical to avoid confirmation bias, I can only guess at the practicalities involved and the effect they would have on Stereophile's work flow and mission.
Frankly, testing high output impedance power amps is a waste of time because their performance is so weird and dependent on their operational environment. One of the basic requirements of a reasonable tset is that the item you are testing has properties that are somewhat stable and reasonably independent of their operational context. There's no reason to test something that changes dramatically every time you change its operational environment because your test results have zero generality.
I think you're treading on dangerous ground here! Not because I don't personally think you're right, or subscribe to that design philosophy, but because I don't have any objective evidence that high output impedance power amplifiers don't have special sonic qualities that make them desirable, as some audiophiles report. Just my sense that the whole business is a crock which is based on intentionally introduced coloration, and ABX tests which find no audible difference in the linear range when frequency response aberrations are equalized out. Unfortunately, that's not proof.
That article is very old news. Les Leventhal was dealt with on a professional basis by The Audio Engineering Society.
How so? That's news to me.
The basic problem is that there are no known reliable bias-controlled listening test methodologies that produce the results that magazines like Stereophile need to justify their existence and support their credibility. Many people here understand that they are basically technically useless advertising vehicles. They do contain some valid technical information, but its there to create a perception of factuality that they generally lack.
This I think involves speculation on your part regarding the motives of Stereophile's staff. I try in general not to impute motive, because for the most part such imputations aren't falsifiable, which makes them a prime tool of confirmation bias. As in "She only ran in front of that truck to save the baby because she wanted to be on the evening news." Not, alas, much of an exaggeration of the way people use this tool to conclude whatever they want.
I do disagree with what I regard as excessive subjectivity on the part of the audiophile press, and a refusal to consider at least some controls. As Stereophile's founder said, "As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me, because I am associated by so many people with the mess my disciples made of spreading my gospel." The irony here is that as often as not, ABX testing does confirm the existence of subjective differences, as why wouldn't it, given the current state of the art. Absent the questionable, there would still be lots to compare, including both differences that do show up in ABX tests and the gray area of subtle differences that practical blind testing may not reveal.
BTW, did you see Jim Austin's "As We See It" in the March issue of Stereophile? He concludes, "Yet a science-based activity without scientific constraints, in which the only distinction among tweaks that appear to be nothing more than snake-oil, well-designed amplifiers, and speakers with good dispersion characteristics are the vicissitudes of personal aural experience, makes me uncomfortable. I find myself craving some certainty, if only to put a little more space between the creations of a skilled audio designer and, say, a jar of petty rocks."
Not necessarily John Atkinson's personal views, as he made clear in another forum, but he did see fit to publish the essay, not, I think, something he would have done had his intention been merely to defend Stereophile's methodology from all comers.
Leventhal's suppositions are no more credible than the whining about confusion and stress. The real problem is that we have a large segment of the audio industry that is based on fallacious assertions and bad logic.
Here I would have to ask for supporting evidence. I have experienced fatigue and stress in forced-choice tests, and I know others have reported the same experience, including IIRC some testers on HA. What I can't do is objectify my personal experience, in terms of whether or not it had an effect on my ability to reliably detect subtle differences. It would be possible to design an experiment to test that hypothesis, but, I think, difficult to conduct one owing to the large number of trials required for a statistically meaningful result.
Frankly, I wish it were otherwise, because as you pointed out there's a lot of snake oil out there. But as I said, I can demonstrate that there are easily audible distortions that won't show up on an ABX test of practical length. I have no reason to believe that there aren't subtler forms of distortion that have such characteristics. Unfortunately, most of the candidates I can think of are loudspeaker-related, e.g., the frequency-dependent harmonic distortion that afflicts some ribbons, and so aren't suitable, since different loudspeakers can always be ABXed. One could intentionally introduce intermittent distortion to demonstrate the point, but it would either be trivially obvious or represent artificial conditions.
At the end of the day, I'm left with the impression that sighted testing over-reports (and sometimes under-reports) differences and introduces bias, while ABX testing can potentially under-report them and can't prove that they don't exist. Not very satisfying, I'm afraid, since it leaves a gray region about which nothing rigorous can be said.
I have experienced fatigue and stress in forced-choice tests, and I know others have reported the same experience, including IIRC some testers on HA.
Yes, it is quite difficult to look for differences where seemingly none exist. Keep this in mind next time you read something from a placebophile talking about how music instantly comes to life and the other horseshit you will read in Stereophile and on forums which do not require objective methods when discussing sound quality.
You've read this insightless piece of garbage, haven't you, Josh358:
http://www.stereophile.com/features/308mp3cd (http://www.stereophile.com/features/308mp3cd)
???
I'm afraid you've lost me! I believe I already said that I personally wish that Stereophile did more blind testing, though I won't go so far as to say "it is really not that hard if there was a will," because I'm not personally familiar with the practical constraints under which the magazine operates.
Constraints? Well, it does sell advertising to lots of manufacturers and vendors who might not take kindly to 'no difference supported' DBT results....
But the audiophile press' antipathy to DBT seems as much philosophical -- bordering on religious -- as anything else. I have heard Mr. Atkinson tell the tale of his Damascene conversion from DBT advocate (though I'm not sure how deep that ever ran) to one who seems to find DBT quite beside the point. IIRC he set up a DBT between an amp* he liked and another amp that was cheaper. The DBT didn't support an audible difference, so he went with the cheaper amp. Some time later he found himself dissatisfied, swapped in the tube amp, and all was bliss again. So to him, that meant DBTs aren't useful.
Now to me, the thing to do would be to re-do a DBT *then*, when presumably one is sensitized to the faulty 'sound' of the 2nd amp. (Indeed, audiophiles are forever complaining that the DBTs they read about didn't allow enough time for the listener to 'learn' the sound of the devices under test. One would think a clear published demonstration of this need, by Mr. Atkinson, would be a boon to their argument.) I asked Mr. Atkinson why he didn't try that - his response, more or less, was that he didn't see the point.
(* an interesting twist here: I seem to recall that amp #1 was a tube amp, and #2 was an SS amp -- so a priori, a positive DBT result would not be as remarkable as SS vs SS)
Yes, it sounds like a second DBT would have been useful and interesting. Though I can't really comment on why JA decided that another test wasn't worthwhile, whether he thought the result would be the same, whether he thought it was unnecessary from a personal perspective, what have you. He'd have to answer that.
In theory, it's possible to conduct an ABX test of any length, but in practice, there are practical constraints. So if lengthy listening does in fact have benefits, they will tend to be lost in an ABXing regime.
Nonsense.
Another argument against ABX testing is that its better suited to basic psychometric evaluations with test signals than it is to music, which is a complex signal and puts great demands on short term memory. This would also be one of the arguments for lengthy testing, since long term memory has a greater capacity than short term memory.
Again, nonsense.
it can't demonstrate with certainty that such differences exist, or how common they are.
"Flying spaghetti monster" argument duly noted.
If you care to make a substantive argument, grounded in fact or logic, I'll be delighted to respond. I'm afraid that the word "nonsense" does not qualify.
Your reference to flying spaghetti monsterism, that is, the claim that assertions can be made on the basis of faith alone, is in error: rather obviously, I was referring to two limitations of statistical analysis, rather than asserting that because such differences cannot be proven or enumerated by statistics they can be said to exist or not exist. That would indeed by flying spaghetti monsterism -- whichever conclusion one reached.
I have experienced fatigue and stress in forced-choice tests, and I know others have reported the same experience, including IIRC some testers on HA.
Yes, it is quite difficult to look for differences where seemingly none exist. Keep this in mind next time you read something from a placebophile talking about how music instantly comes to life and the other horseshit you will read in Stereophile and on forums which do not require objective methods when discussing sound quality.
You've read this insightless piece of garbage, haven't you, Josh358:
http://www.stereophile.com/features/308mp3cd (http://www.stereophile.com/features/308mp3cd)
???
Believe me, I take reports of such differences with a grain of salt. Those whose opinions I do consider plausible (though not proven to my satisfaction) are those who point out that differences of the sort we're discussing are subtle. But I can't say that listener fatigue is a consequence of listening for non-existent differences. There just is no evidence for or against that. Personally I find it difficult to concentrate on the many aspects of sound reproduction simultaneously while worrying about a floating reference. You may not listen that way. I do. Whether it's valid or not, I don't know, but I've found ABX tests on musical material fatiguing, even when the results were positive.
I think JA made the mistake of letting the measurements speak here, rather than referencing tests which show that psychoacoustic phenomena mask most compression artifacts on musical material. OTOH, I agree with his argument about not using lossy codecs in high fidelity reproduction. After all, DBT's tell us that they can be audible. I'm not bothered by the artifacts in high bit rate MP-3's, but that doesn't mean that others aren't. As long as they're hearing real artifacts and not imagined ones, I don't have any argument with their preference (or the preference of those who think they're just fine).
If you care to make a substantive argument, grounded in fact or logic, I'll be delighted to respond. I'm afraid that the word "nonsense" does not qualify.
I expect substantiation from you since you are the one putting up the theory. The burdendoes not fall on me to disprove it. Perhaps you have some verifiable psychological studies to present?
It should be simple to answer this one at least:
what was the title of your presentation in Seattle?
There was no formal title. I was introduced to the audience with the words "And now John Atkinson of Stsreophile will play some of his high-resolution recordings."
John Atkinson
Editor, Stereophile
Even if that were the case (can you supply some objective proof ?), the closest thing to a title would be the way your presentation was promoted in your fine publication as stated in the lead post in this thread .... and that is clearly not neutral in terms of leading expectation bias ..... is it?
Another argument against ABX testing is that its better suited to basic psychometric evaluations with test signals than it is to music, which is a complex signal and puts great demands on short term memory. This would also be one of the arguments for lengthy testing, since long term memory has a greater capacity than short term memory.
That's one of Bob Stuart's arguments. I don't think it stacks up...
All I can say is that I reached this conclusion independently, on the basis of my own experience. I can't, as I think I said, demonstrate conclusively that it has a practical effect.
In both critiques, it seems apparent to me that the "good" way of doing something like existing subjective testing, but with double-blind statistical certainty, is firstly to do double blind A/B testing where there's no limit on time or source material. Listen to A for a month if you want - play all the music you own (OK, that would take a few years for some of us, but you get the point). Then have a go at B. Then try direct comparisons if you wish.
Greynol just made that suggestion in a different context. I think it's a good one. If you can't distinguish two pieces of gear in a DBT after becoming intimately familiar with them, knowing what recordings make them go gaga, etc., I doubt very much you can distinguish them under any circumstances.
They're all examples of low probability events, which per Leventhal's analysis would require an inordinate number of trials for a 95% CI.
They're not though. If a slightly broken speaker cone reveals problems with solo piano music, you run 16 ABX trial with piano music - not one each with each random CD you one. If there's a highly intermittent fault (even a subtle one) you pick X=A or X=B when you hear it - you don't pick anything until you do.
This gets back to the above. I agree. If I made some notes when I heard a piano recording distorting, and then ABX'd that segment, it would be obvious. However, this presupposes knowledge of the audible flaw. Most ABX tests don't begin with that presupposition.
The fact that magazine rarely is ever do double blind A/B testing, never mind the X part, speaks volumes IMO.
The fact that you could actually do a full standard sighted test, and then ABX whatever you found to be most revealing - AND THEN PEOPLE USUALLY FAIL - is also quite strange. Or not.
Or, as you say, not. ;-)
It's strange to me only because I've always taken my own perceptions with a grain of salt. Some people seem oblivious to the problem of listening bias. They can't quite wrap their minds around the fact that what they think they hear, they may not hear. And, honestly, I find that puzzling.
There's no great excuse against blind A/B - certainly not where all the testing happens 9-5 in the magazine's office. Obviously at home there are other issues, but it's not insurmountable.
I don't really think you should publish reviews if you can't manager to do them properly. But then, it's a free market. There are people who want to pay to read flawed test reports.
Cheers,
David.
Flawed, sure, but don't you think that it's an imperfect world? I have to depend on my unaided ears for much of what I do, because it's impractical to do anything else. In practice, I find myself using a combination of listening, theory, reviews, research and scuttlebutt -- and even then, I'm frequently uncertain. This despite the fact that I avoid tweaks of questionable utility, such as esoteric cables. Merely moving a pair of speakers to another part of the room can be a challenging exercise; the differences are obvious, but has the speaker become more or less accurate? To me, those who are glib about their ability to analyze such differences lose credibility, unless they're specialists who might be expect to have special knowledge -- an acoustician, say, in the case of speaker positioning.
In practice, I find reading a magazine like Stereophile a bit like trying to form impressions of audio equipment, in that I can glean what I believe is useful information from it, but only by applying the sort of painstaking skepticism I apply to my assessments of audio equipment. One of the criteria that I sometimes use is that of independent verification of perception. If, for example, I think the frequency response of a particular loudspeaker is elevated in the presence range, and a reviewer notes that that the frequency response of that particular loudspeaker is elevated in the presence range, it gives me faith both in my perceptions and those of the reviewer. It's for this reason more than any other that I don't dismiss out of hand subjective reviews -- I have noticed such concordances, in circumstances that made it unlikely that the confirmation bias was systematic. Conversely, if a critic makes statements that I believe are inconsistent with physics and engineering -- claiming for example that a power cord has sonic characteristics that one would normally associate with a mechanical transducer such as a microphone or speaker -- I conclude that he's very probably full of it and move on.
Which isn't to say that I don't wish I could ABX my stereo with an orchestra! But more often than not, we have to make do.
If you care to make a substantive argument, grounded in fact or logic, I'll be delighted to respond. I'm afraid that the word "nonsense" does not qualify.
I expect substantiation from you since you are the one putting up the theory. The burdendoes not fall on me to disprove it. Perhaps you have some verifiable psychological studies to present?
OK, here are the statements to which you objected:
"In theory, it's possible to conduct an ABX test of any length, but in practice, there are practical constraints. So if lengthy listening does in fact have benefits, they will tend to be lost in an ABXing regime."
Do you object to my assertion that there are practical constraints on the length of ABX tests (or number of samples/participants, I wrote hastily)? I had taken that as a given.
I didn't maintain that lengthy listening does have benefits, so there's no assertion there. I merely said "If it does." I'm not aware of any substantive evidence one way or the other. So no assertion to defend, merely a hypothesis to be tested. You yourself gave an example of one such possible test in your discussion of JA's amplifier DBT. A better testing regime might include a statistical analysis of ABX results before and after a period of lengthy listening. Presumably, if familiarity improves performance, it will show up in superior ABX scores. One might infer such an outcome from the verified improvement in consistency of judgment after listeners were trained in speaker analysis at Harman, and the observation, also made by Olive and Toole, that in the absence of training the ability to make consistent judgments increases with listening experience, but only as a likelihood, owing to the substantial differences in conditions.
"Another argument against ABX testing is that its better suited to basic psychometric evaluations with test signals than it is to music, which is a complex signal and puts great demands on short term memory. This would also be one of the arguments for lengthy testing, since long term memory has a greater capacity than short term memory."
Presented merely as an argument. I happen to think it likely, as I said in another post, based on my own experience with analytical listening and ABX testing. But I don't pretend to be able to prove or disprove it scientifically: it's merely a hypothesis. I welcome you to do either, if you have any substantive theoretical or practical evidence -- or even just an intuition, in which case your hunch will be on equal footing with mine.
Potentially, if these common objections could be debunked -- or, if shown to be true, circumvented through improved testing regimes -- it would bolster the case for double blind testing as a reliable arbiter of audibility.
Yes, you're right, you've posed what is just a theory as a hypothetical. My curt response is due to the fact that we've seen it before, though usually by people as an excuse for their complete unwillingness to perform a double-blind test. I think the theory has absolutely no merit, myself. If you're straining to choose A or B when being presented X, then there is probably nothing special about either A or B that would cause either to stand out. If after lifting the cover changes the way it sounds to you, then you have your answer.
We already know how easy it is to tell someone how to detect the "evils of mp3" objectively. It is also quite easy to tell someone what expectation bias is. This can be done without having to involve someone in double-blind tests. Perhaps I may have missed it, so can you give an example of JA or Stereophile doing either aside from defensive posturing against performing such methods, ever?
I appreciate that you attempt to take an objective approach in your decision making. Unfortunately not everyone is as keen about this as you and I think it's a shame that JA is doing little, if anything to improve the situation. Rather it is readily apparent to many of us that he would rather people stay in the dark. Perhaps you can give me a hypothetical postulation as to why he would allow reviews to be published that glorify hideously expensive power and mains cables. Regarding reviews that agree with your sighted observations, have you ever read nonsensical subjective claims from a reviewer with whom you agreed on something that didn't seem nonsensical? If yes, how do you square this? If it were me, I would think it would lead me to question my own objectivity (or lack thereof).
If I made some notes when I heard a piano recording distorting, and then ABX'd that segment, it would be obvious. However, this presupposes knowledge of the audible flaw. Most ABX tests don't begin with that presupposition.
I think most do. Most people who use ABX use it when they think they hear a problem, but aren't sure - or when they're sure they hear a problem, but want to prove it to a scientific audience. It's rare to run an ABX test when you don't think you hear a problem. It does happen, but I don't think it's the more common scenario.
In practice, I find reading a magazine like Stereophile a bit like trying to form impressions of audio equipment, in that I can glean what I believe is useful information from it, but only by applying the sort of painstaking skepticism I apply to my assessments of audio equipment. One of the criteria that I sometimes use is that of independent verification of perception. If, for example, I think the frequency response of a particular loudspeaker is elevated in the presence range, and a reviewer notes that that the frequency response of that particular loudspeaker is elevated in the presence range, it gives me faith both in my perceptions and those of the reviewer. It's for this reason more than any other that I don't dismiss out of hand subjective reviews -- I have noticed such concordances, in circumstances that made it unlikely that the confirmation bias was systematic. Conversely, if a critic makes statements that I believe are inconsistent with physics and engineering -- claiming for example that a power cord has sonic characteristics that one would normally associate with a mechanical transducer such as a microphone or speaker -- I conclude that he's very probably full of it and move on.
You can take that approach, but it's very frustrating. Given that the very same people will report audible problems which (to me) are gross, subtle, and non-existent in
exactly the same terms, it's not that useful. Given that we have a rigorous scientific approach that can cut out all the nonsense with fairly surgical precision, it seems strange to reject it. It's like
wanting to believe in magic so much that you ignore the bit where someone shows you how the trick was done.
Cheers,
David.
In reality creating such an impulse graph that Apodizing marketing uses is only possible with mathematics that result in a bent phase response (that alone was an audiophile no-no back a while), a bad frequency response and what i am not sure about, it must introduce TONS of aliasing.
Imaging. Not aliasing. And no, there is no need for such a filter to introduce 'TONS' of it. Not even minimal amounts. The Meridian filter doesn't. SoX and Izotope, when properly configured, don't. The Ayre filter does,
but then the, already raped-over, term 'apodisation' doesn't apply there at all.
(Could we drop the silly terminology and simply discuss what particular filters are and do?)
Yes, the Meridian filter introduces non-linear phase shift. And yes this used to be a no-no.
There hasn't been a lot of rigorous study on the audibility of such, but surveying what is available leads me
to believe that some phase distortion is not a bad thing, provided it stays out of the mid-band and evolves monotonically with frequency.
As for frequency-magnitude response. Is ruler-flat to 18kHz with a small drop at 20kHz 'bad'?
Understanding masking is one of those things that we who get Zwicker and Fastl et al have over your average Joe the high end audiophile or rick the recording engineer.
Let it be noted that understanding stuff is declared useful in discussions like these.
"in the context of Peter Craven's so-called "apodizing" reconstruction filter. This eliminates the conventional linear-phase "ringing" at the Nyquist frequency but introduces minimum-phase "ringing" at a slightly lower frequency."
This is false. What figures 1 and 2 of the relevant Stereophile review of the Meridian 808i.2 show is not an elimination of linear phase ringing, but rather a reapportionment and what seems to be a significant increase in total energy in the ringing of the so-called Apodizing filter.
Looks to me like typical high end engineering -
In fact JA's description is quite correct. If you had understanding of AA and AI filters (ha!) you would know that for a given cutoff and steepness the summed energy in the pre and post lobes is a constant, and that nulling the pre-ringing would double the post-ringing energy.
And as for the slight drop in ringing frequency, well, that touches the core of Craven's thinking, and how he came to adopt the term 'apodising' in the first place.
As has been pointed out before JA uses what has been for decades the high end's *perfect* obfuscating smokescreen - he avoids engaging in reliable listening tests. If you can fool enough of the paying customers some of the time and fool yourself all of the time... ;-)
For years I have been wondering about your animosity towards JA. You heap the (perceived) sins of an entire industry onto an individual. And yet, the members of that industry, including its customers, seem quite happy with the state of affairs. I find your outright hatred, frankly, boring.
The Meridian engineers report that it's linear phase up to about 18kHz. The phase response does bend above 18kHz to achieve their desired result.
I have a rough copy of the Meridian filter at home (thanks to internet-based espionage, although no laws were violated and no sentient beings were harmed).
Phase shift starts well below 18kHz, but stays within remarkable bounds. They did some clever engineering there.
"The individual impulses are added together and the pre and post ringing is canceled out."
If there's signal content at/around the filter's cut-off frequency, the ringing certainly isn't cancelled out.
Yes, this is quite correct.
The sampling theorem proves that for a linear phase reconstruction filter approximating Sinc(t) all the ringing disappears ...
... for a band-limited input signal.
But most ADCs today employ half-band linear-phase AA filters, which inject their own pre- and post-ringing into the captured signal. This ringing can only be removed by filtering again at a lower cutoff frequency, and with a minimum-phase response. Which brings us back to Meridian.
Further, the AA filters being half-band restricts their attenuation at Nyquist to (multiples of) 6dB. So the signal isn't really bandlimited and the AI-filter side ringing is excited as well.
I think we suspect that pre-ringing is somewhat more audible than post-ringing but not to the extent that several times the amplitude could be overcome.
Ask JJ. He has the data. Although not at 22kHz.
Plus, the Apodized wave rings at an approx 10% lower frequency which is also a slight disadvantage.
See above. It is the crux of this all.
That would be the interesting experiment, to move the filters' corner frequencies down until one or the other (hopefully not both!) becomes audible, to see which one becomes audible first at the highest frequency.
Interesting, but not necessarily relevant. The frequency band above, say, 12kHz hits the cochlea in a spot that does its processing significantly different than lower frequencies. Hence perception above/below 12kHz differs as well. Ask JJ. He knows that pre-ringing in the mid-band is bad. But not necessarily at the edge of audibility.
IOW follow a linear phase filter with a 22.05 KHz corner frequency by a minimum phase filter with a 20 KHz corner frequency to attenuate the linear phase filter's ringing at 22.05 KHz. The apodizing filter need not provide a ton of attenuation, since the liner phase filter still reduces out-of-band responses.
If the aim is to annihilate the linear phase pre-response then the apodising filter really needs a ton of attenuation. Further, the linear phase filter referred to is in the ADC, and the apodiser is in the DAC. If suppression of images is required (and I can't see why it suddenly wouldn't be so), then the minimum phase filter has to supply, again, a ton of attenuation. These aren't exactly secrets.
Have a nice day.
That article is very old news. Les Leventhal was dealt with on a professional basis by The Audio Engineering Society.
How so? That's news to me.
You might want to check out the JAES which is where the Leventhal controvesy played out directly and in-person, not Stereophile which was reporting their biased view.
Stereophile has a vested interest in magnifying any controversy relating to the results of reliable listening tests since the results of virtually all realible listening tests are so different from their day-to-day fare.
One big turning point in the history of reliable listening tests was the ready availability of software and files for people to use to do their own listening tests.
The basic problem is that there are no known reliable bias-controlled listening test methodologies that produce the results that magazines like Stereophile need to justify their existence and support their credibility. Many people here understand that they are basically technically useless advertising vehicles. They do contain some valid technical information, but its there to create a perception of factuality that they generally lack.
This I think involves speculation on your part regarding the motives of Stereophile's staff.
I think that the interpreation above of what I said is reasonble given what I wrote, but that is not what I had in mind when I wrote it. I was not thinking about their motives,. I was thinking about the reactions of people who are well informed about subjective testing. We are generally mystified by what they write about many things given our own personal expereinces. We are mystified by their apparent lack of desire to subject their opinions and beliefs to reliable listening tests. Some of us once thought as they still appear to do. As they say: "Listening is believing:", but we believe that we have vastly reduced the effcts of common bases on our listening tests. They seem to allow strong extraneous influences intrude on their listening. We thinkg that we have had good sucess in vastly reducing those extraneous influences.
In my view Leventhal was barking up the wrong tree. First off, you can increase the sensitivity of a listening test by simply running more trials. For those of us who routinely do a lot of trials because we are investigating a goodly number of alternatives, all this whining about getting tired gets old fast. If you don't do reliable listening tests your need to investigate a number of alternatives is reduced because you will probably get the results you seek more quickly because your desire for the results you seek contaminate the evaluation.
BTW, did you see Jim Austin's "As We See It" in the March issue of Stereophile? He concludes, "Yet a science-based activity without scientific constraints, in which the only distinction among tweaks that appear to be nothing more than snake-oil, well-designed amplifiers, and speakers with good dispersion characteristics are the vicissitudes of personal aural experience, makes me uncomfortable. I find myself craving some certainty, if only to put a little more space between the creations of a skilled audio designer and, say, a jar of petty rocks."
I see that as an strage statement. First off, what's this "speakers with good dispersion characteristics are the vicissitudes of personal aural experience" stuff? Who said that? So chalk up 1 (one) straw man. Next, what is this "Yet a science-based activity without scientific constraints"?. Yet another straw man. And so on. I feel no need to seriously address what seems to be nothing more than debating-trade prattle. I've always been more interseted in the study of technology than the study of rhetoric.
Leventhal's suppositions are no more credible than the whining about confusion and stress. The real problem is that we have a large segment of the audio industry that is based on fallacious assertions and bad logic.
Here I would have to ask for supporting evidence.
Been there, done that but any reasonable recital of it vastly exceeds the domain of one post to HA.
I have experienced fatigue and stress in forced-choice tests, and I know others have reported the same experience, including IIRC some testers on HA.
Does this surprise you? Remember that most of the sighted evaluations you have done or heard of are so weak that they generally fail to even meet the basic definition of being a proper test. Its not the forced choice that stresses you, its the fact that we don't tell you the *right answer* before, during and after the test. Sighted evaluations are like multiple choice tests with a cheat sheet right there on the desk in front of you. So not being five the right answers stresses you out? Who'd a thunk?
What I can't do is objectify my personal experience, in terms of whether or not it had an effect on my ability to reliably detect subtle differences. It would be possible to design an experiment to test that hypothesis, but, I think, difficult to conduct one owing to the large number of trials required for a statistically meaningful result.
You're not doing an apples-to-apples comparison. You're comparing a demo (thats all that a sighted evaluation is is a demo) to a real live test.
"in the context of Peter Craven's so-called "apodizing" reconstruction filter. This eliminates the conventional linear-phase "ringing" at the Nyquist frequency but introduces minimum-phase "ringing" at a slightly lower frequency."
This is false. What figures 1 and 2 of the relevant Stereophile review of the Meridian 808i.2 show is not an elimination of linear phase ringing, but rather a reapportionment and what seems to be a significant increase in total energy in the ringing of the so-called Apodizing filter.
This is false. In the given example, the apodizing filter does not merely re-aopportion. It changes the frequency of the ringing and significantly increases its total energy.
In fact JA's description is quite correct. If you had understanding of AA and AI filters (ha!) you would know that for a given cutoff and steepness the summed energy in the pre and post lobes is a constant, and that nulling the pre-ringing would double the post-ringing energy.
The above seems to state that the steepness and cutoff frequency remained the same. Graphical analysis shows that the cutoff frequency dropped by about 10%. I can't comment on the steepness due to the limitations of the data. Graphical analysis suggests a significant increase (doubling) in the energy in the ringing.
And as for the slight drop in ringing frequency, well, that touches the core of Craven's thinking, and how he came to adopt the term 'apodising' in the first place.
Noting your ability to dismiss evidence that conflicts with your beliefs... We now agree that the corner frequency dropped significantly, which falsifies your previous two statements, I can live with that! ;-)
As has been pointed out before JA uses what has been for decades the high end's *perfect* obfuscating smokescreen - he avoids engaging in reliable listening tests. If you can fool enough of the paying customers some of the time and fool yourself all of the time... ;-)
For years I have been wondering about your animosity towards JA.
It's not animosity, its understanding of his situation and an offer of help for his obvious problems with the relevant facts.
You heap the (perceived) sins of an entire industry onto an individual.
That seems to be a pretty bizarre allegation. How do you reliably measure which sins I perceive and where I place the blame? Are you in possession of everything that I've written and posted? Have you audited it thoroughly and accurately? Here's a test for you - give an estimate of the total number of words I've posted on the sins of the high end and compare that with the ones that blamed them on on just John Atkinson. Note that even just the words quoted above show signfiicant attribution of techical shortcmings to someone else. The text you quoted falsifies your claim that I blame only Atkinson!
What you see here is due the fact that Atkinson posts here and stands up for himself like a man. If I go over to Harley's forum and try to talk with him, he often doesn't even reply, and then he waits a little while and deletes my posts. That puts Atkinson way ahead of Harley in my estimation. I give Atkinson kudos for parting ways with Harley long ago.
And yet, the members of that industry, including its customers, seem quite happy with the state of affairs.
That seems to be be a presumption on your part. For example, we don't know how much turnover there is of customers in the high end audio business. At least I don't. Do You? Got a reliable source and numbers?
I can infer something from the fact that Stereophile's subscription base seems to be relatively stable while they seem to be promoting the magazine to new customers quite aggressively...
Are you in possession of everything that I've written and posted? Have you audited it thoroughly and accurately?
What, all the rec.audio.opinion threads included?!?!
I think I've just had a brief vision of hell.
OK, here are the statements to which you objected:
"In theory, it's possible to conduct an ABX test of any length, but in practice, there are practical constraints. So if lengthy listening does in fact have benefits, they will tend to be lost in an ABXing regime."
Do you object to my assertion that there are practical constraints on the length of ABX tests (or number of samples/participants, I wrote hastily)? I had taken that as a given.
The important question is whether or not there are unusual practical constraints that significantly reduce the sensitivity of ABX tests. By unusual, I mean peculiar to just ABX tests or just peculiar to bias controlled tests.
As far as length of test, length of sample, length of trial, number of participants etc go, the answer is no. There are many successful ABX tests that carried these issues well beyond practicality. I'm talking days, even weeks. Furthermore, we know for sure that at least some of those variables have optimal values that are relatively short, IOW measured in seconds. We know for sure that for some of those variables, longer is very bad.
One of the best articles about these issues is "Flying Blind" by Tom Nousiaine which should Google up pretty well. It was published in Audio Magazine, if memory serves.
The one big, hard to solve problem with listening tests that I see is that test sensitivity can be highly dependent on the musical selection being used for the test. Pick the right one and the test can be a slam dunk. Pick the wrong one and it can be mission impossible. But this is true of all listening tests that are actually tests. The only reason why people don't run into this problem in sighted evaluations is the fact that they aren't really tests.
Here is a problem to be wary of - don't fault ABX for problems that actually apply to all or most good listening tests. There are a ton of such problems, but you don't find out about them until you stop doing just demos and actually start doing tests.
Are you in possession of everything that I've written and posted? Have you audited it thoroughly and accurately?
What, all the rec.audio.opinion threads included?!?!
I think I've just had a brief vision of hell.
What, me ask a trick question? ;-)
Imaging. Not aliasing. And no, there is no need for such a filter to introduce 'TONS' of it. Not even minimal amounts. The Meridian filter doesn't. SoX and Izotope, when properly configured, don't. The Ayre filter does,
but then the, already raped-over, term 'apodisation' doesn't apply there at all.
I admit i don´t know to much about how these filters work exactly and only gave back what i have read around the net lately.
As you can see there are many claims and explanations why someones system is better. May it be Ayre or Meridian or some forum mathematicans that add some strategies.
Like i quoted before Benchmark Media even states this pre-ringing in 44.1 kHz is most likely a non-issue!
For one or two things i don´t have the need to know exactly how it works.
If it works it should only make the recordings sound different that have pre-ringing.
If it changes every recording in sound it fails its purpose and acts like a DSP effect imho.
Since modern recordings often come from high resolution masters some studio engineer may already have used a non-linear resampler to produce the 44.1 master. What if such a recording gets apodized again?
Edit: And i still would like to see someone with generated files does a serious abx. I can´t.
That would be the interesting experiment, to move the filters' corner frequencies down until one or the other (hopefully not both!) becomes audible, to see which one becomes audible first at the highest frequency.
Interesting, but not necessarily relevant. The frequency band above, say, 12kHz hits the cochlea in a spot that does its processing significantly different than lower frequencies. Hence perception above/below 12kHz differs as well. Ask JJ. He knows that pre-ringing in the mid-band is bad. But not necessarily at the edge of audibility.
I suspect that you don't realize that the experiment I suggested would shed light on exactly the issue you raised. Hence your rather nonsensical judgement of "not necessarily relevant". In fact: Highly relevant.
It should be simple to answer this one at least:
what was the title of your presentation in Seattle?
There was no formal title. I was introduced to the audience with the words "And now John Atkinson of Stsreophile will play some of his high-resolution recordings."
Even if that were the case (can you supply some objective proof ?)...
What would you require in the way of "objective proof"? The sessions weren't recorded in any manner.
the closest thing to a title would be the way your presentation was promoted in your fine publication as stated in the lead post in this thread .... and that is clearly not neutral in terms of leading expectation bias ..... is it?
The event wasn't promoted in Stereophile, purely on the magazine's website - see http://www.stereophile.com/content/music-m...attle-wednesday (http://www.stereophile.com/content/music-matters-6-seattle-wednesday) - and in the promotion done by Seattle retailer Definitive Audio (pictured in the Stereophile Web item). Frankly, I think you and "krabapple" are clutching at skeptical straws here. Sometimes a cigar is just a cigar. :-)
John Atkinson
Editor, Stereophile
But the audiophile press' antipathy to DBT seems as much philosophical -- bordering on religious -- as anything else. I have heard Mr. Atkinson tell the tale of his Damascene conversion from DBT advocate (though I'm not sure how deep that ever ran)...
That seems like a cheap shot, Krabapple. My formal education was in hard science, my first jobs were in hard science, and in fact one of the first editorial comments I had published in Hi-Fi News (in April 1979) was a criticism of subjective reviewing and the lack of blind testing.
...to one who seems to find DBT quite beside the point. IIRC he set up a DBT between an amp* he liked and another amp that was cheaper. The DBT didn't support an audible difference, so he went with the cheaper amp. Some time later he found himself dissatisfied, swapped in the tube amp, and all was bliss again. So to him, that meant DBTs aren't useful.
No, it meant that that blind test, for unknown reasons, was not sufficiently sensitive to detect small but possibly real audible differences between the amplifiers. Note that I did not set up the test nor did I choose the amplifiers under test. I was involved purely as a listener, a test subject. And as you yourself have pointed out, one of the amplifiers in that late-1970s test was a tube design with a high output impedance, so may well have introduced frequency response differences that were large enough to be audible.
Now to me, the thing to do would be to re-do a DBT *then*, when presumably one is sensitized to the faulty 'sound' of the 2nd amp. (Indeed, audiophiles are forever complaining that the DBTs they read about didn't allow enough time for the listener to 'learn' the sound of the devices under test. One would think a clear published demonstration of this need, by Mr. Atkinson, would be a boon to their argument.) I asked Mr. Atkinson why he didn't try that - his response, more or less, was that he didn't see the point.
I explained to you back then that I was merely choosing an amplifier to use in my system, just like any other audiophile. How many HA posters, for example subject their audio purchases to rigorous double-blind testing before making a decision on what to buy?
My long-term dissatisfaction with the inexpensive amplifier I purchased as a result of the blind test results was real, even there was no sign that it was not performing correctly. I replaced it with one that it turned out I preferred the sound of under conditions of normal use. Achieving that was my goal as a consumer and the goal was achieved. It is very easy more than 30 years later for you to say, "well you should have done more listening tests" but back then I didn't see the need.
John Atkinson
Editor, Stereophile
The event wasn't promoted in Stereophile, purely on the magazine's website
*sigh*
Whether or not it was promoted in your publication is irrelevant. Anyone who read the news promo linked in the first post was told your presentation will demonstrate the "evils of mp3".
That seems like a cheap shot, Krabapple. My formal education was in hard science, my first jobs were in hard science, and in fact one of the first editorial comments I had published in Hi-Fi News (in April 1979) was a criticism of subjective reviewing and the lack of blind testing.
Your actions speak louder than your past credentials, I'm afraid. It is a shame that you've turned your back on this.
My long-term dissatisfaction with the inexpensive amplifier I purchased as a result of the blind test results was real, even there was no sign that it was not performing correctly.
I'm sure it was, and I am just as sure that the dissatisfaction had absolutely nothing to do with the way it actually sounded. Had someone secretly changed the innards of your new amplifier with those of the old one you would have never noticed.
the closest thing to a title would be the way your presentation was promoted in your fine publication as stated in the lead post in this thread ...
The event wasn't promoted in Stereophile, purely on the magazine's website
Whether or not it was promoted in your publication is irrelevant.[/quote]
I was correcting the poster's comment that the event had been promoted in the magazine. Please don't read anything more into my statement than that.
Anyone who read the news promo linked in the first post was told your presentation will demonstrate the "evils of mp3".
Except that, as I have written before in this thread, with the actual example that included the lossy encoded sections, the audience was told that they were going to be listening to a 24-bit, 88.2kHz recording by Philip Hobbs of Linn Records. If they had any expectation bias, I wold have thought it would operate in the opposite direction to your implication. It is entirely possible that the audience, if they were even aware of the "evils of MP3" mentioned in the Stereophile website piece, assumed that _that_ demonstration would occur later in the event.
John Atkinson
Editor, Stereophile
I was correcting the poster's comment that the event had been promoted in the magazine.
...and conveniently left the more pertinent questions unanswered.
If they had any expectation bias, I wold have thought it would operate in the opposite direction to your implication.
The assumed direction of the outcome does not change the fact that the promo was released. Once again, you've chosen to ignore the glaring error by focusing on the window dressing.
Not one of the so far here presented hypothetical shortcomings of ABX tests has been anywhere near convincing.
But can we really neglect the following:
What if human beings are of such nature, that their sense of hearing is not independent, such that shear association of a sound with an item can change the actual perception of that sound?*
Let the item(s) be two identical amplifier circuits, one boxed in a cheap, neon-green plastic housing, connected with bell wires, the other one boxed in a nice, solid metal-encloure, with finish and design by a lead industrial-design professional, connected with the same bell wires, but encapsulated in expensive looking lining, with gold plated connectors, and laid out very neatly, almost rectilinear. Both setups measure absolutely identical in every respect.
Now do three rounds of testing:
1. A sighted test. Actual sources are truthfully reported.
I think, most of us share the expectation, that a representative share of the population would produce very significant results in favor of the expensive looking setup's sound. Me probably included.
2. A sighted test. Actual sources are not truthfully reported.
If you properly randomize the procedure, the 2. test will not reveal any significant correlation between the actual sound source and the perceived quality of its output, but still a strong correlation between believed source and perceived quality.
3. A double blind test.
Again there will be no correlation between actual sound source and actual perception of differences.
Thus one might prematurely conclude that there are no practically relevant differences between both amps. But that's really not entirely true. It is only true for situations, where people are blindfolded or actively lied to. In actual practice both amps still produce a very significant difference with regard to how their sound is perceived. And 'practice' here means a mode of listening which comes closest to how people actually listen to music at home, for example while watching their tube amps glow.
There are two options now. Let the tube amp proponent enjoy his comfy listening sessions, belief, and honest perception that his gear sounds superior (it does to him). Or try to hammer an abstract (though valid) truth into his head, that he is fooling himself, and that you could prove it by blindfolding him. Both are valid approaches. But I do not think that those, who fight for option 2 at the edge of hatred, should think that reason w.r.t. a better world is 100% on their side here.
* hint: the question is rhetorical
Except that, as I have written before in this thread, with the actual example that included the lossy encoded sections, the audience was told that they were going to be listening to a 24-bit, 88.2kHz recording by Philip Hobbs of Linn Records. If they had any expectation bias, I wold have thought it would operate in the opposite direction to your implication. It is entirely possible that the audience, if they were even aware of the "evils of MP3" mentioned in the Stereophile website piece, assumed that _that_ demonstration would occur later in the event.
John Atkinson
Editor, Stereophile
How long is HA going to entertain this farce?
Except that, as I have written before in this thread, with the actual example that included the lossy encoded sections, the audience was told that they were going to be listening to a 24-bit, 88.2kHz recording by Philip Hobbs of Linn Records. If they had any expectation bias, I wold have thought it would operate in the opposite direction to your implication. It is entirely possible that the audience, if they were even aware of the "evils of MP3" mentioned in the Stereophile website piece, assumed that _that_ demonstration would occur later in the event.
John Atkinson
Editor, Stereophile
Oh, come on, John.
You know (and everyone else knows) that you left scientific methods out because of the nice quantity of money you earn selling audiophile nonsense to the masses.
You can't have that, and ABX testing (or any other scientific method of testing). You are perfectly aware of that.
You are not stupid man; you are aware that proper ABX testing of lossless vs. lossy would give you unwanted results.
So, to further advertise "high fidelity" (the term is now very perverted), you do tests like these.
Their results can be interpreted in your favor, and in the favor of companies who pay commercials in your magazine.
OK, that is business, I understand that. You are making money on other people's ignorance. You sold yourself long ago.
But please, don't try to sell your methods as valid scientifical methods, and don't try to tell people that the results are perfectly valid.
They are not. You know that.
Except that, as I have written before in this thread, with the actual example that included the lossy encoded sections, the audience was told that they were going to be listening to a 24-bit, 88.2kHz recording by Philip Hobbs of Linn Records. If they had any expectation bias, I wold have thought it would operate in the opposite direction to your implication. It is entirely possible that the audience, if they were even aware of the "evils of MP3" mentioned in the Stereophile website piece, assumed that _that_ demonstration would occur later in the event.
John Atkinson
Editor, Stereophile
How long is HA going to entertain this farce?
I don't see it as a farce. I see it as an opportunity for people who are weak or undecided to hear directly from one of the more transparent people in high end audio how they hide from science, reason, and self-awareness.
But the audiophile press' antipathy to DBT seems as much philosophical -- bordering on religious -- as anything else. I have heard Mr. Atkinson tell the tale of his Damascene conversion from DBT advocate (though I'm not sure how deep that ever ran)...
That seems like a cheap shot, Krabapple.
The whole anecdote is a cheap shot. I would like to title it "John Atkinson believes that all blind listening tests sound the same". It was a poorly-prepared test. The first ABX test I ever did was miles further down the road, and we've carried blind tests far further in the following decades.
My formal education was in hard science, my first jobs were in hard science, and in fact one of the first editorial comments I had published in Hi-Fi News (in April 1979) was a criticism of subjective reviewing and the lack of blind testing.
The world is full of people who actually staued with hard science who have no clue about doing good listening tests, some of whom don't get what the fuss is all about even when it is properly explained to it. Most of them don't have a much to lose by changing their minds as you do, John.
If you really don't get blind testing at this late stage John, then you sold your brain down the river. If you do get it, but you're hiding it, then you sold your soul. We've all seen what you were and are getting paid, so the only possible discussion is over what you exchanged to obtain it.
Noting your ability to dismiss evidence that conflicts with your beliefs... We now agree that the corner frequency dropped significantly, which falsifies your previous two statements, I can live with that! ;-)
If you had the decency of quoting me correctly, instead of putting your own words on my keyboard (really, go up and read it all again!), you would see that I never contested that drop in cutoff frequency. Quite the contrary.
Oh, and when it comes to digital filters like these I don't have believes. I have understanding.
If this thread follows the succession of others like this, we should soon see Arnold drowning the thread in continued ranting against John Atkinson and the industry as a whole and page long nitpicking with WernerO, to make it clear that Arnold's not reading WernerO carefully is much less significant than Arnold's strong feeling to fight side by side with the truth.
Noting your ability to dismiss evidence that conflicts with your beliefs... We now agree that the corner frequency dropped significantly, which falsifies your previous two statements, I can live with that! ;-)
If you had the decency of quoting me correctly, instead of putting your own words on my keyboard (really, go up and read it all again!), you would see that I never contested that drop in cutoff frequency. Quite the contrary.
Oh, and when it comes to digital filters like these I don't have believes (sic). I have understanding.
It's pretty clear Werner that English isn't your first language, so I'll be charitable and attribute your loud and obnoxious false perceptions of technical errors on my part to your language problems.
You did make one very good point though, and that the Ayre apodizing filters generate idealistic impulse responses at the cost of substandard rejection of out-of-band signals. This is pretty strange because they make a point of claiming that they solved that problem here:
Ayre MP White Paper (http://www.ayre.com/pdf/Ayre_MP_White_Paper.pdf)
This paper says that the QB-9 uses their new Apodizing filters, so the Stereophile review of the product can be taken as an example of their performance.
The Stereophile tests of the Ayre QB-9 show a completely different story here;
Stereophile Ayre USB DAC tech tests (see figure 13) (http://www.stereophile.com/content/ayre-acoustics-qb-9-usb-dac-measurements)
For those to whom the interpretation of this evidence is not obvious, let me point out the following:
The DAC is being excited with digital test tones at 19 and 20 KHz which will naturally create images at 24.1 KHz and 25.1 KHz. In a well-designed DAC the reconstruction filter will pass the 19 and 20 KHz tones with minimal attenuation and will sharply attenuate the images at 24.1 and 25.1 KHz. Figure 12 shows a credible job of attenuation by the linear phase *Measure* filter. The images at 24.1 and 25.1 KHz are about 90 dB down. Figure 13 which appears to show the effects of the Anodizing *Listen* filter which provides only about about 9 dB worth of attenuation.
It is important to understand that these undesired artifacts at 24.1 and 25.1 KHz are practically innocuous in a good audio system. Of course if you have signficiant amounts ot nonlinear distortion at very high frequencies in the signal chain, they could cause more audible distortion because nonlinear distortion implies intermodulation distortion, and a significant intermodulation diffrerence tone could be created at 1 KHz. Not filtering the images might increase these distortion products by something like 6 dB. because the in-band tones at 19 and 20 KHz would have already created similar distortion products at the same frequency.
I see no evidence that Ayre's alleged listening tests were anything but sighted evaluations. Barring reliable evidence their whole schtick shapes up in my view as being an trite attempt to sell an (at best) a $200 USB DAC for $2,500. It apparently appeals to people whose favorite aerobic exercise is suspending disbelief. ;-)
It's pretty clear Werner that English isn't your first language, so I'll be charitable and attribute your loud and obnoxious false perceptions of technical errors on my part to your language problems.
...as if this isn't obnoxious??? Considering that most of your posts contain glaring typos, you shouldn't be throwing stones. WTF does "staued" mean?
Atkinson's procedure reminds me of nothing so much as the "triple-blind" test procedure promoted by the crank Clark Johnsen a while back. It was described in issue 17 of The Audio Critic (http://www.theaudiocritic.com/back_issues/The_Audio_Critic_17_r.pdf) by Jeff Corey in an article entitled "The Blind Misleading the Blind:"
Another gentleman introduced himself with, "I attended Harvard and have a degree in physics. I invented the triple-blind method." [This was Clark Johnsen. See also the preceding article.—Ed.] Then he gave me a preprint. I avidly scanned the preprint because the only triple-blind method I ever heard of was the old joke about doing a double-blind experiment and then losing your data.
As it became apparent upon reading the paper, this triple-blind procedure was a joke, too. The new twist turned out to be that the subjects were not told that they were being tested. They were presented with recorded material in "reverse polarity" and then, after a selection played for a while, "the operator (so as not to cause undue suspicion) winked at the subject and spoke words to the effect, 'We'll do that again.'" Then the operator switched the wires to the correct polarity and "stayed out of sight to fufill the criterion of a blind (unseen) operator." After the test was over, the operator returned to the listening room, gave the subject a response sheet and asked, "Did you hear a difference?"
This exemplifies one of my favorite experimental psychology exam questions. Given an example of a hopelessly flawed study, how many flaws can the student find? An adequate scholar would list:
1. No informed consent was obtained.
2. Reverse polarity was always presented first. A confounding order effect is likely, as shown in a later paper by Tom Nousaine.
3. The operator always knows the condition being presented. Verbal or nonverbal cues may bias the subject against the first presentation (with the operator present). The next cue that something different (maybe even better) is coming up is given away by the wink. Finally, being away during the second presentation does not count. The key point is that the potentially biased experimenter was present during the collection of the data.
4. There is no guarantee that the procedure of reversing polarity did not affect other salient features of the presentation, such as the exact decibel level, which can affect subjective ratings (Nousaine's paper again).
While polarity may have a real effect, this study did not demonstrate anything warranting the author's conclusions. The triple-blind design is a hoax. When I questioned the presenter later, he replied that he was a "great actor" and could disguise his bias from the subject. Nod, nod, wink, wink. Or as the Pythons put it, "A nod is as good as a wink to a blind wombat."
You did make one very good point though, and that the Ayre apodizing filters generate idealistic impulse responses at the cost of substandard rejection of out-of-band signals. This is pretty strange because they make a point of claiming that they solved that problem here:
Ayre MP White Paper (http://www.ayre.com/pdf/Ayre_MP_White_Paper.pdf)
This paper says that the QB-9 uses their new Apodizing filters, so the Stereophile review of the product can be taken as an example of their performance.
The Stereophile tests of the Ayre QB-9 show a completely different story here;
Stereophile Ayre USB DAC tech tests (see figure 13) (http://www.stereophile.com/content/ayre-acoustics-qb-9-usb-dac-measurements)
For those to whom the interpretation of this evidence is not obvious, let me point out the following:
The DAC is being excited with digital test tones at 19 and 20 KHz which will naturally create images at 24.1 KHz and 25.1 KHz. In a well-designed DAC the reconstruction filter will pass the 19 and 20 KHz tones with minimal attenuation and will sharply attenuate the images at 24.1 and 25.1 KHz. Figure 12 shows a credible job of attenuation by the linear phase *Measure* filter. The images at 24.1 and 25.1 KHz are about 90 dB down. Figure 13 which appears to show the effects of the Anodizing *Listen* filter which provides only about about 9 dB worth of attenuation.
Just to clarify these measurements, which Mr. Krueger seems to have confused, the filter whose high-frequency intermodulation is shown in fig.13 is not an "apodizing" filter. Its response at 22.05kHz is -6dB rather than a complete null. The "Listen" filter is thus a slow roll-off type, and as with all such filters, has poor image rejection. This filter's impulse response is shown as fig.3 in the linked page - there is almost no "ringing," which the engineers at Ayre have told me they feel to be a subjectively worthwhile trade-off against the filter's poor image rejection. By contrast, fig.12 shows the HF intermodulation spectrum with Ayre's "Measure" filter. This appears to be a clone of the Craven "apodizing" filter and demonstrates excellent image rejection. The impulse response of this filter is shown as fig.2.
John Atkinson
Editor, Stereophile
Just to clarify these measurements, which Mr. Krueger seems to have confused, the filter whose high-frequency intermodulation is shown in fig.13 is not an "apodizing" filter.
The page I cited says:
"Measure is said to resemble the minimum-phase "apodizing" filter used by Meridian in their 808i.2 CD player, which I reviewed in April; the Listen filter has been optimized by Ayre's engineering team to give much of the benefit of the apodizing filter, but with less ringing."
I see nothing but senseless haggling over a shading of the meaning of words.
Does anybody really care whether the "listen" filter is an apodizing filter or one that has been optimized to give much of the benefit of an apodizing filter?
The point I was trying to make apparently still stands, and that is that the "listen" filter does a very substandard job of rejecting out-of-band artifacts.
Does anybody really care whether the "listen" filter is an apodizing filter or one that has been optimized to give much of the benefit of an apodizing filter?
When one continuously and boisterously attends to the splinters in the eyes of others, one may care. It may also help to become aware of a certain log.
Just to clarify these measurements, which Mr. Krueger seems to have confused, the filter whose high-frequency intermodulation is shown in fig.13 is not an "apodizing" filter.
The page I cited says:
"Measure is said to resemble the minimum-phase "apodizing" filter used by Meridian in their 808i.2 CD player, which I reviewed in April; the Listen filter has been optimized by Ayre's engineering team to give much of the benefit of the apodizing filter, but with less ringing."
Correct. I wrote "gives the benefits," by which I meant (and I thought obvious from the context), the _subjective_ benefits ( if there were any).
I see nothing but senseless haggling over a shading of the meaning of words.
Does anybody really care whether the "listen" filter is an apodizing filter or one that has been optimized to give much of the benefit of an apodizing filter?
I was addressing the incorrect criticism you and others had made that "apodizing" filters necessarily have poor anti-imaging performance. For you to point to the poor anti-imaging behavior of Ayre's "Listen" fllter as an example of this criticism was misleading because it is _not_ an "apodizing" filter and is therefore not typical. As I pointed out to you, it is a slow-rolloff filter that is 6dB down at Nyquist. It is more like the filters offered by products from Wadia, Sony, Pioneer, and Luxman in that respect.
John Atkinson
Editor, Stereophile
Yes, you're right, you've posed what is just a theory as a hypothetical. My curt response is due to the fact that we've seen it before, though usually by people as an excuse for their complete unwillingness to perform a double-blind test. I think the theory has absolutely no merit, myself. If you're straining to choose A or B when being presented X, then there is probably nothing special about either A or B that would cause either to stand out. If after lifting the cover changes the way it sounds to you, then you have your answer.
We already know how easy it is to tell someone how to detect the "evils of mp3" objectively. It is also quite easy to tell someone what expectation bias is. This can be done without having to involve someone in double-blind tests. Perhaps I may have missed it, so can you give an example of JA or Stereophile doing either aside from defensive posturing against performing such methods, ever?
I appreciate that you attempt to take an objective approach in your decision making. Unfortunately not everyone is as keen about this as you and I think it's a shame that JA is doing little, if anything to improve the situation. Rather it is readily apparent to many of us that he would rather people stay in the dark. Perhaps you can give me a hypothetical postulation as to why he would allow reviews to be published that glorify hideously expensive power and mains cables.
The irony is that JA is now getting flak from both sides, someone just lit into him on the critic's asylum for publishing Jim Austin's "As We See It." I gather from what John says in that thread that he agrees at least in part with Jim's analysis, but, really, I think questions about his beliefs and editorial philosophy are best directed to him. All I can do is express my personal wish that Stereophile took a more skeptical approach to controversial claims. This would, I believe, bolster both the objectivist and subjectivist opinions -- I say both because I believe it would tend to undermine some of the more controversial subjectivist claims about power cords and the like, while supporting more credible claims of audible differences -- after all, their are plenty of *positive* ABX tests.
Regarding reviews that agree with your sighted observations, have you ever read nonsensical subjective claims from a reviewer with whom you agreed on something that didn't seem nonsensical? If yes, how do you square this? If it were me, I would think it would lead me to question my own objectivity (or lack thereof).
I guess the short form answer is that I don't have faith in my own objectivity. My reaction towards a critic's description of a subtle phenomenon is invariably one of skepticism, as it would be if I'd made the observation myself. And really, even some that aren't so subtle or in dispute, particularly where subjective ranking is involved. Forex, I often find myself wondering about the effect of price on preferences, remember doing so even before I read about Toole's experiment.
On the other hand, Toole and others have established that some listeners are better at detecting differences than others, and I think there's a lot of reason to suppose that training, experience, and maybe innate capability can have a positive effect on the ability to do so. I've seen that first hand, in my own work -- the skill with which an experienced audio mixer or colorist, say, can identify and fix a flaw that would take me much longer to identify and figure out, and that someone who's had no technical experience in audio or video would be at a loss to identify and fix at all. So it's possible for me to respect a particular critic's ability to listen astutely, while also recognizing that no one is immune to listening bias.
I haven't really followed the audiophile critics closely since the Enid Lumley/Tice Clock era began, but still, I'm periodically impressed by the ability of some critics to hear and identify the same things I did, without supposing that it confers upon them some sort of immunity from the biases which afflict us all.
The problem from my perspective has always been telling what's bias and what isn't. I try to use all the means at my disposal to do that, including independent observations, theory, DBT's, and measurements, but I'm painfully aware that I don't always know, sometimes even when DBT's have been performed, as in those conflicting reports on the audibility of 44.1 downconversion.
The irony is that JA is now getting flak from both sides, someone just lit into him on the critic's asylum for publishing Jim Austin's "As We See It." I gather from what John says in that thread that he agrees at least in part with Jim's analysis, but, really, I think questions about his beliefs and editorial philosophy are best directed to him. All I can do is express my personal wish that Stereophile took a more skeptical approach to controversial claims.
The irony compounded:
"I felt the "scientific" side of the Great Debate had been under-represented in recent issues, so when Jim offered me his essay, I accepted it. " -- JA responding in that AudioAsylum thread (http://www.audioasylum.com/cgi/t.mpl?f=critics&m=54189)
So, JA felt his magazine had been coming up a bit short on science recently, and made sure his audiophile public got a bit of redress.
Was the scientific side represented adequately in JA's road show 'demonstrations'?
That is the question that this thread's about. Not whether this or that filter implementation can be heard.
Was the scientific side represented adequately in JA's road show 'demonstrations'?
I felt so. (And why the scare quotes around the word "demonstrations"? Regardless of people's opinions, they were indubitably demonstrations.)
John Atkinson
Editor, Stereophile
I think that the interpreation above of what I said is reasonble given what I wrote, but that is not what I had in mind when I wrote it. I was not thinking about their motives,. I was thinking about the reactions of people who are well informed about subjective testing. We are generally mystified by what they write about many things given our own personal expereinces. We are mystified by their apparent lack of desire to subject their opinions and beliefs to reliable listening tests. Some of us once thought as they still appear to do. As they say: "Listening is believing:", but we believe that we have vastly reduced the effcts of common bases on our listening tests. They seem to allow strong extraneous influences intrude on their listening. We thinkg that we have had good sucess in vastly reducing those extraneous influences.
In that, we agree.
In my view Leventhal was barking up the wrong tree. First off, you can increase the sensitivity of a listening test by simply running more trials. For those of us who routinely do a lot of trials because we are investigating a goodly number of alternatives, all this whining about getting tired gets old fast. If you don't do reliable listening tests your need to investigate a number of alternatives is reduced because you will probably get the results you seek more quickly because your desire for the results you seek contaminate the evaluation.
The questions here are what is practiced, what is practical, what is confusing, and the degree (if any) to which long-term listening allows one to hear differences that aren't apparent during short-term listening. Which is a lot of questions. There are also questions of experimental design, e.g., ABXes of 44.1 kHz sampling that are in fact ABXes of 44.1. kHz sampling using certain algorithms with certain source material under certain conditions, and produce contradictory results as a consequence.
Until I see a more satisfactory and *objectively confirmed* answer to these dilemmas, objectivity requires me to acknowledge the existence of a gray area in DBT, just as I do in subjective listening. I wish that that weren't the case, since it would end a certain degree of puzzlement on my part.
BTW, did you see Jim Austin's "As We See It" in the March issue of Stereophile? He concludes, "Yet a science-based activity without scientific constraints, in which the only distinction among tweaks that appear to be nothing more than snake-oil, well-designed amplifiers, and speakers with good dispersion characteristics are the vicissitudes of personal aural experience, makes me uncomfortable. I find myself craving some certainty, if only to put a little more space between the creations of a skilled audio designer and, say, a jar of petty rocks."
I see that as an strage statement. First off, what's this "speakers with good dispersion characteristics are the vicissitudes of personal aural experience" stuff? Who said that? So chalk up 1 (one) straw man. Next, what is this "Yet a science-based activity without scientific constraints"?. Yet another straw man. And so on. I feel no need to seriously address what seems to be nothing more than debating-trade prattle. I've always been more interseted in the study of technology than the study of rhetoric.
His point was that subjective testing doesn't distinguish between snake oil and good engineering, and that engineering, a scientific endeavor, should not primarily be judged by unscientific means. I'm not sure why you object to it, since it would seem to be concordant with your own beliefs.
I have experienced fatigue and stress in forced-choice tests, and I know others have reported the same experience, including IIRC some testers on HA.
Does this surprise you? Remember that most of the sighted evaluations you have done or heard of are so weak that they generally fail to even meet the basic definition of being a proper test. Its not the forced choice that stresses you, its the fact that we don't tell you the *right answer* before, during and after the test. Sighted evaluations are like multiple choice tests with a cheat sheet right there on the desk in front of you. So not being five the right answers stresses you out? Who'd a thunk?
That's not why it fatigues me, and that hasn't been my experience. I find any A/B comparison of complex material fatiguing, sighted or not, since I have to compare dynamically changing program material and keep multiple factors in short-term memory. I find a comparison of dynamically-changing musical material with a floating X doubly confusing. There are differences which the brain detects easily and with facility, e.g., frequency response differences of the magnitude known to be audible; we have evolved to do this, are in fact constantly adjusting perceived frequency response in almost real time to separate the effects of comb filtering and the HRTF. But there are other audible differences which I find much more difficult to detect in a short-term comparison, despite the fact that I can, given enough time, successfully ABX them. And some easily phenomena simply *will not* show up on a statistically significant ABX test of practical length, conducted on material with which the listener is unfamiliar. I've described a few of them here.
What I can't do is objectify my personal experience, in terms of whether or not it had an effect on my ability to reliably detect subtle differences. It would be possible to design an experiment to test that hypothesis, but, I think, difficult to conduct one owing to the large number of trials required for a statistically meaningful result.
You're not doing an apples-to-apples comparison. You're comparing a demo (thats all that a sighted evaluation is is a demo) to a real live test.
Not at all. None of what I have said here is based on observations made in sighted testing. I have never alleged that *any* sighted observation is objectively reliable, and I've made no assertions to the effect that a specific sighted observation is reliable when an ABX observation is not, merely raised the possibility that it could be. I know of no cases in which I would make such an assertion about a specific observation except where my technical knowledge leads me to identify an experimental error which is inconsistent with existing theory and research, e.g., the supposed inaudibility (in the case of unshaped dither) of a 16-bit bit depth reported in Meyer and Moran. But their error is perhaps indicative of some of the pitfalls that can occur when we fail to be critical enough of experimental results.
What I am saying is that it is, as a matter of basic science, impossible to prove that a negative ABX result is valid, and that, in practice, there are reasons why an ABX test could potentially under-report minor differences, as well as experimental errors. I think there is a gray region where no definitive assessment can be made, and consider it important to avoid being carried away by suppositions of objectivity that are unjustified, whether they apply to subjective testing or DBT.
The important question is whether or not there are unusual practical constraints that significantly reduce the sensitivity of ABX tests. By unusual, I mean peculiar to just ABX tests or just peculiar to bias controlled tests.
As far as length of test, length of sample, length of trial, number of participants etc go, the answer is no. There are many successful ABX tests that carried these issues well beyond practicality. I'm talking days, even weeks. Furthermore, we know for sure that at least some of those variables have optimal values that are relatively short, IOW measured in seconds. We know for sure that for some of those variables, longer is very bad.
One of the best articles about these issues is "Flying Blind" by Tom Nousiaine which should Google up pretty well. It was published in Audio Magazine, if memory serves.
The one big, hard to solve problem with listening tests that I see is that test sensitivity can be highly dependent on the musical selection being used for the test. Pick the right one and the test can be a slam dunk. Pick the wrong one and it can be mission impossible. But this is true of all listening tests that are actually tests. The only reason why people don't run into this problem in sighted evaluations is the fact that they aren't really tests.
Here is a problem to be wary of - don't fault ABX for problems that actually apply to all or most good listening tests. There are a ton of such problems, but you don't find out about them until you stop doing just demos and actually start doing tests.
It's not so much that I'm faulting ABX, as that I'm faulting any test that suffers from a particular constraint, sighted or unsighted. One of those is, as you say, the selection of source material. In practice, I find that this is particularly troubling with some resonant phenomena. It's no secret for example that some ribbons have serious distortion spikes at certain frequencies.
Was the scientific side represented adequately in JA's road show 'demonstrations'?
I felt so. (And why the scare quotes around the word "demonstrations"? Regardless of people's opinions, they were indubitably demonstrations.)
John Atkinson
Editor, Stereophile
I'm dubious that they demonstrated what they were meant to. I'm even more dubious that there was anything scientific about them.
Your own scare quotes around 'scientific' in your audioasylum reply suggest a certain dubiety as well.
What I am saying is that it is, as a matter of basic science, impossible to prove that a negative ABX result is valid, and that, in practice, there are reasons why an ABX test could potentially under-report minor differences, as well as experimental errors. I think there is a gray region where no definitive assessment can be made, and consider it important to avoid being carried away by suppositions of objectivity that are unjustified, whether they apply to subjective testing or DBT.
It is impossible for statistical results, which is to say scientific results, to be 'definitive' in the vernacular sense ('absolutely' right or wrong)...there is always built-in proviso that says, 'this could be a mistake'. Looking for a 'definitive assessment' in that sense would be naive. Dismissing a weight of evidence because it can't be 'definitive' in that sense would be too.
Audiophiles -- whether published in magazines or online -- constantly make audible difference reports that, based just on what's known of the physiology human hearing, psychoacoustics, and physics itself, are at best unlikely to be true and at worst laughably preposterous. That's before we even get to discussing ABX test results as part of the evidence.
Even Stereophile makes a distinction of sorts here...differences between cables and transports are blithely accepted as audible, at least to those with ears of gold, but the effects of 'Intelligent Chips' aren't. Though I wonder if they've polled Jason Serinus on that.
It is impossible for statistical results, which is to say scientific results, to be 'definitive' in the vernacular sense ('absolutely' right or wrong)...there is always built-in proviso that says, 'this could be a mistake'. Looking for a 'definitive assessment' in that sense would be naive. Dismissing a weight of evidence because it can't be 'definitive' in that sense would be too.
Audiophiles -- whether published in magazines or online -- constantly make audible difference reports that, based just on what's known of the physiology human hearing, psychoacoustics, and physics itself, are at best unlikely to be true and at worst laughably preposterous. That's before we even get to discussing ABX test results as part of the evidence.
Even Stereophile makes a distinction of sorts here...differences between cables and transports are blithely accepted as audible, at least to those with ears of gold, but the effects of 'Intelligent Chips' aren't. Though I wonder if they've polled Jason Serinus on that.
Can't argue with that! And this, I think, is an area in which DBT can be extremely useful. Not because DBT's can prove that a difference is inaudible, but because they can demonstrate to a high degree of probability that a difference, if it does exist, is either so subtle as to be on the edge of perception, or intermittent. Or, for that matter, that a controversial difference is audible.
What I am saying is that it is, as a matter of basic science, impossible to prove that a negative ABX result is valid, and that, in practice, there are reasons why an ABX test could potentially under-report minor differences, as well as experimental errors. I think there is a gray region where no definitive assessment can be made, and consider it important to avoid being carried away by suppositions of objectivity that are unjustified, whether they apply to subjective testing or DBT.
I then question whether you get science.
(1) *Every* finding of science is provisional, and is merely the best result we've obtained up until that point.
(2) Every experiment that is done based on the best understandings that were generally known and accepted when it was done *is valid* in terms of the results that can be reasonably inferred from it.
When we started out doing ABX tests in 1975 or so, our initial tests were very sophisticated for the day, and often still very sophisticated compared to what a lot of audiophiles, recording engineers and audio journalists do, but very crude by our modern standards.
There's currently no need to worry about tests with negative outcomes because in almost every case because we now know how to structure the test so that a relevant positive outcome can be obtained. For example, instead of stopping with trying to hear the artifacts generated by a preamp, we can hook a 100 preamps in series in a sensible way and see if 100 times the artifacts generated by one preamp is audible and then shorten the string of preamps until the artifacts are no longer audible. Or, we can know which artifacts of the preamp are most likely to be heard and determine how the preamps generation of that artifact compares with the foreknown or experimentally developed threshold for that artifact. These aren't just philosophical ideas and speculations, they represent things that have been done.
Many people have done the necessary DBTs so that is is meaningful for them to say that they can always positively ABX MP3s with a bitrate of 32 kbps, that they have no problems finding music that is ABXable at 128 kbps, but at 320 kbps things get really, really hard or impossible for them. HA had gone through many generations of people who say that they have a magical recording that makes the difference between 16/44 and 24/192 like night and day. So they upload the magic recording and AFAIK we have always found that the easy pickings go away when we clean their experiment up.
Some of us also know that the artifacts associated with good interconnects are minuscule compared to everything I've mentioned so far. It's not much of a logical leap to dismiss them based on both inference and for some of us, direct empirical evidence.
We are not saying that absence of evidence is evidence of absence, we're saying that we know that the threshold is here and we can quantify situations in terms of which side of the threshold they are on.
The idea of limits beyond which human performance can not transcend is valid. While there has been a slow progression to miles run faster than 4 minutes, the evidence at hand does not reasonably predict a Mile being run in less than 1 minute during this century. Link to info about world's records for running a mile (http://en.wikipedia.org/wiki/Mile_run_world_record_progression)
We now know that every issue of Stereophile and TAS and the rest of their ilk is tantamount to a claim that the mile has been run in a few seconds, or less. ;-)
The questions here are what is practiced, what is practical, what is confusing, and the degree (if any) to which long-term listening allows one to hear differences that aren't apparent during short-term listening. Which is a lot of questions. There are also questions of experimental design, e.g., ABXes of 44.1 kHz sampling that are in fact ABXes of 44.1. kHz sampling using certain algorithms with certain source material under certain conditions, and produce contradictory results as a consequence.
What makes you think that long-term listening is at all revealing of differences? Tom Nousaine has shown that a pool of experienced listeners cannot even reliably detect 4% distortion under long-term listening conditions (link (http://www.nousaine.com/pdfs/Flying%20Blind.pdf)), even though 4% distortion can be discerned with 100% accuracy under formal ABX test conditions (as shown later in the same article). Clearly, this suggests that ABX tests are in fact
highly revealing of non-imaginary differences.
to make it clear that Arnold's not reading WernerO carefully is much less significant than Arnold's strong feeling to fight side by side with the truth.
It is obvious now that Arnold feels no reason to read my words carefully, or quote them
properly, as English is not my first language. If he read them properly, he would probably
learn a thing or two about these digital filters. Or engineering in general. His loss, not mine.
In the 20+ years I've been acquainted with this character I have not learned anything useful
from him. I guess I always was too thick to read and comprehend his English.
The reasonable thing to do would be to ignore him altogether, but apparently the more warped side of my psyche appreciates the entertainment once in a while. It's cheaper than a zoo ticket.
(http://img210.imageshack.us/img210/9080/arny.gif) (http://img210.imageshack.us/i/arny.gif/)
This filter's impulse response is shown as fig.3 in the linked page - there is almost no "ringing," which the engineers at Ayre have told me they feel to be a subjectively worthwhile trade-off against the filter's poor image rejection.
Dear John,
the above story is misleading. True, there is no pre-ringing when the 'listen' filter is excited with a digitally-generated impulse. This is obvious and predicable.
But it has nothing to do with reality, as the digital impulse is an illegal signal: it is not band-limited and should not exist in the sampled data space. (Or at least carry a 'handle with care' label.)
The only impulse response of relevance is the total system's response, analogue-in to analogue-out, including the ADC's AA filter.
And most anti-alias filters, be they part of the ADC's silicon or a software resampler used during mastering, still have linear-phase half-band architectures with pre-ringing and 6 or 12dB attenuation at Nyquist.
Concatenate these with the Ayre 'listen' filter and there still will be massive ringing before and after the impulse. Try it.
It is obvious now that Arnold feels no reason to read my words carefully, or quote them improperly, as English is not my first language.
Not so. What I'm saying is that due to a combination of your demonstrated desire to attack and destroy me personally, combined with your hindered ability to interpret what I write, you've created a blood battle where none need to exist.
The rest of your post is completely reprehensible and shows an utter lack of respect, so I won't further degrade this thread by quoting it or trying to address your unfortunate situation.
The questions here are what is practiced, what is practical, what is confusing, and the degree (if any) to which long-term listening allows one to hear differences that aren't apparent during short-term listening. .
What makes you think that long-term listening is at all revealing of differences? Tom Nousaine has shown that a pool of experienced listeners cannot even reliably detect 4% distortion under long-term listening conditions (link (http://www.nousaine.com/pdfs/Flying%20Blind.pdf)), even though 4% distortion can be discerned with 100% accuracy under formal ABX test conditions (as shown later in the same article). Clearly, this suggests that ABX tests are in fact highly revealing of non-imaginary differences.
Good point. Tom's "Flying Blind" article was the result of years of experience by the ABX team in SE Nichigan combined with an open effort to obtain the best possible sensitivity to small differences and subtle effects by any means,
The first step in developing just about anything is developing a criteria that reliably produces positive and negative results. What I'm talking about here is the concept of falsifiability.Wikipedia article about Falsifiability (http://en.wikipedia.org/wiki/Falsifiability)
Sighted evaluations generally fail the test of Falsifiabilty since they almost always produce positive results no matter what is being tested. Interestingly enough a good explanation of why this is true can be found in Stereophile:
Stereophile article that explains what sighted listening evaluations actually test (http://www.stereophile.com/asweseeit/505awsi)
Despite what Curl seems to suggest, there's no need to turn away from science to understand the mode of action of the Intelligent Chip. In the March 2005 "Industry Update" (p.24) I described a recent study that used functional magnetic resonance imaging (fMRI) to map the brain activity of people drinking Coca-Cola and people drinking Pepsi. When participants knew they were drinking Coke, two things happened: their preference was affected, and parts of the brain were activated that are thought to be involved in emotional and affective influences on behavior. These areas of the brain, the researchers noted, "may participate in recalling cultural information that biases preference judgments." There's nothing to be done; perception is inherently subjective. Whether it's taste or sound, perception has as much to do with the brain as it does with the sensory organs.
There's some interesting science behind the Intelligent Chip, but it is not especially novel, and it's happening between our ears, not between the chip and the CD. JA admits that, years ago, he was at first convinced by Enid Lumley's demonstration of the pizza-box tripod. "When she did the test, I did hear the difference," he wrote to me in a recent e-mail correspondence. "On my own, no difference, which I ascribed to Enid's powers of persuasion."
"Not everyone is comfortable with that level of introspection. Some people would rather see angels, aliens, and forces unknown to science than admit that much of the crap that goes down inside our brains is sordid, subjective, or just plain embarrassing. It's human nature, I suppose; plenty goes on inside my own head that I'd rather not share with the neighbors, though I do try to keep tabs on it myself. "
This filter's impulse response is shown as fig.3 in the linked page - there is almost no "ringing," which the engineers at Ayre have told me they feel to be a subjectively worthwhile trade-off against the filter's poor image rejection.
Dear John,
the above story is misleading. True, there is no pre-ringing when the 'listen' filter is excited with a digitally-generated impulse. This is obvious and predicable.
But it has nothing to do with reality, as the digital impulse is an illegal signal: it is not band-limited and should not exist in the sampled data space. (Or at least carry a 'handle with care' label.)
Yes indeed. The reason I use the single sample, with its inherent >Nyquist bandwidth, is purely to reveal the filter's impulse response in isolation.
The only impulse response of relevance is the total system's response, analogue-in to analogue-out, including the ADC's AA filter.
And most anti-alias filters, be they part of the ADC's silicon or a software resampler used during mastering, still have linear-phase half-band architectures with pre-ringing and 6 or 12dB attenuation at Nyquist.
Concatenate these with the Ayre 'listen' filter and there still will be massive ringing before and after the impulse. Try it.
I agree. The elimination of the complete system ringing is the elegance of the Craven filter, which has a null at 22.05kHz. As the Ayre Listen filter is a slow-rolloff type and is down only 6dB at Nyquist, it does almost nothing to eliminate the system ringing you describe. I did try to be clear in my postings that I was referring to Ayre's claims of subjective superiority of this filter. While I have my opinion on the difference between Ayre's Listen filter and their Measure filter, the latter an apodizing design, it would not be appropriate to offer that opinion on HA, given that I haven't tested that opinion with ABX testing, as required by the forum's rules.
Sorry to see you fall foul of what Arny Krueger calls his "debating trade." Best to withdraw gracefully, as you have done.
John Atkinson
Editor, Stereophile
It's not so much that I'm faulting ABX, as that I'm faulting any test that suffers from a particular constraint, sighted or unsighted. One of those is, as you say, the selection of source material. In practice, I find that this is particularly troubling with some resonant phenomena. It's no secret for example that some ribbons have serious distortion spikes at certain frequencies.
But then you have to fault every possible scientific test, don't you? They all suffer from constraints, every single one of them.
Of course that's why scientists repeat tests many times and refuse to accept them until other scientists have repeated them with the same results.
And even they they never report anything as an absolute truth but say things like "The probability of X lying somewhere between Y and Z is 99%", or if they don't then they aren't being scientists when that happens.
Ed
This filter's impulse response is shown as fig.3 in the linked page - there is almost no "ringing," which the engineers at Ayre have told me they feel to be a subjectively worthwhile trade-off against the filter's poor image rejection.
Dear John,
the above story is misleading. True, there is no pre-ringing when the 'listen' filter is excited with a digitally-generated impulse. This is obvious and predicable.
But it has nothing to do with reality, as the digital impulse is an illegal signal: it is not band-limited and should not exist in the sampled data space. (Or at least carry a 'handle with care' label.)
The same goes for the square wave - presented with the implication that a good system would reproduce it as a perfect square wave - whereas only a system with no filter at all would come close!
However, at least measuring the impulse response with a single sample input shows you what the filter
is. Readers can draw their own correct or incorrect conclusions.
And most anti-alias filters, be they part of the ADC's silicon or a software resampler used during mastering, still have linear-phase half-band architectures with pre-ringing and 6 or 12dB attenuation at Nyquist.
Concatenate these with the Ayre 'listen' filter and there still will be massive ringing before and after the impulse. Try it.
True of course. Hence the apodizing filter cutting at a lower frequency to remove some of this...
around we go in circles...
Cheers,
David.
EDIT - DAMN - just made a post almost identical to JA's - my credibility is shot Or at least, John knows how to write for the HA audience as well as he knows how to write for the Stereophile audience.
If this thread follows the succession of others like this
...whether it does or doesn't, I don't think there's much new to learn.
If someone could ABX filters acting at or above 20kHz, then it would be interesting. But that's not going to happen in this thread - or ever?
Cheers,
David.
What I am saying is that it is, as a matter of basic science, impossible to prove that a negative ABX result is valid, and that, in practice, there are reasons why an ABX test could potentially under-report minor differences, as well as experimental errors. I think there is a gray region where no definitive assessment can be made, and consider it important to avoid being carried away by suppositions of objectivity that are unjustified, whether they apply to subjective testing or DBT.
I then question whether you get science.
(1) *Every* finding of science is provisional, and is merely the best result we've obtained up until that point.
Obviously. This, however, does not contradict what I said; rather, it reinforces it. Unless you misconstrued the sense in which I used "definitive"?
(2) Every experiment that is done based on the best understandings that were generally known and accepted when it was done *is valid* in terms of the results that can be reasonably inferred from it.
Again, where is the contradiction? The problem in this case is that the results that are inferred from the experiment are not, typically, the results that can reasonably be inferred from it. It's much easier to falsify the null hypothesis "A and B do not sound the same" than it is to falsify the alternative hypothesis "A and B sound the same." This difference disappears only when one modifies the alternative hypothesis to something like "an unrepresentative sample of listeners were unable to detect a difference between A and B in a forced choice test using these musical selections with this ancillary equipment under these conditions."
When we started out doing ABX tests in 1975 or so, our initial tests were very sophisticated for the day, and often still very sophisticated compared to what a lot of audiophiles, recording engineers and audio journalists do, but very crude by our modern standards.
There's currently no need to worry about tests with negative outcomes because in almost every case because we now know how to structure the test so that a relevant positive outcome can be obtained. For example, instead of stopping with trying to hear the artifacts generated by a preamp, we can hook a 100 preamps in series in a sensible way and see if 100 times the artifacts generated by one preamp is audible and then shorten the string of preamps until the artifacts are no longer audible. Or, we can know which artifacts of the preamp are most likely to be heard and determine how the preamps generation of that artifact compares with the foreknown or experimentally developed threshold for that artifact. These aren't just philosophical ideas and speculations, they represent things that have been done.
I've long advocated tests of that kind, as long as it's remembered that some forms of distortion are cumulative and that this must be taken into account in setting allowable thresholds, that masking can occur in components before and after the component under test, and that, in general, the conclusions are valid only for the conditions of the specific experiment (music selected, etc.).
Many people have done the necessary DBTs so that is is meaningful for them to say that they can always positively ABX MP3s with a bitrate of 32 kbps, that they have no problems finding music that is ABXable at 128 kbps, but at 320 kbps things get really, really hard or impossible for them. HA had gone through many generations of people who say that they have a magical recording that makes the difference between 16/44 and 24/192 like night and day. So they upload the magic recording and AFAIK we have always found that the easy pickings go away when we clean their experiment up.
But it seems that at least one person here can routinely ABX 320 kbit/s MP-3's; I believe he said he detects about three artifacts per song. This is a very good example of the limitations of ABX tests of complex musical material, specifically, the tendency to under-report audible differences (as sighted tests over-report them, at least when that's the direction of the expectation bias as it is among many audiophiles).
Some of us also know that the artifacts associated with good interconnects are minuscule compared to everything I've mentioned so far. It's not much of a logical leap to dismiss them based on both inference and for some of us, direct empirical evidence.
I think we have to be very careful here. I've seen measurements in JAES that indicate that cables can alter high frequency response up towards 20 kHz. The variations were below the known thresholds of audibility, but frequency response variations are of course cumulative and a bias could be audible for that reason with some program material. It is also not to the best of my knowledge established that some can't hear minor response variations that are not shown as audible in the standard psychometric curves.
Consider a study conducted by aliens to determine how accurately humans can determine pitch. A study conducted on westerners would conclude that we are not very good at it. A study conducted on native Chinese speakers would reach a very different result; it's now known that they speak with perfect pitch. An inference made on the basis of a study of westerners who have acquired perfect pitch would also produce a different result than the first study. So I think we have to consider the possibility of hyperacuity, particularly acquired.
OTOH, on a practical level, I feel comfortable in concluding that the likelihood of listener bias here is greater than the likelihood of audible differences that somehow magically disappear in a blind test. I don't bother personally with exotic cables. Not that I did even before the test results, since I saw no plausible engineering justification for them. I'm merely pointing out that I can't rule out the possibility that they can affect the sound.
We are not saying that absence of evidence is evidence of absence, we're saying that we know that the threshold is here and we can quantify situations in terms of which side of the threshold they are on.
The idea of limits beyond which human performance can not transcend is valid. While there has been a slow progression to miles run faster than 4 minutes, the evidence at hand does not reasonably predict a Mile being run in less than 1 minute during this century. Link to info about world's records for running a mile (http://en.wikipedia.org/wiki/Mile_run_world_record_progression)
I quite agree that there are limits on the sensitivity of perception, but see above: I don't think it's always easy to determine where those limits lie, because a random sample may not be adequate to test a skill that has a low probability of occurrence in the regular population. Intuitively, we know that, and try to get around it by using experienced listeners.
We now know that every issue of Stereophile and TAS and the rest of their ilk is tantamount to a claim that the mile has been run in a few seconds, or less. ;-)
There's some of that, no question about it.
The questions here are what is practiced, what is practical, what is confusing, and the degree (if any) to which long-term listening allows one to hear differences that aren't apparent during short-term listening. Which is a lot of questions. There are also questions of experimental design, e.g., ABXes of 44.1 kHz sampling that are in fact ABXes of 44.1. kHz sampling using certain algorithms with certain source material under certain conditions, and produce contradictory results as a consequence.
What makes you think that long-term listening is at all revealing of differences? Tom Nousaine has shown that a pool of experienced listeners cannot even reliably detect 4% distortion under long-term listening conditions (link (http://www.nousaine.com/pdfs/Flying%20Blind.pdf)), even though 4% distortion can be discerned with 100% accuracy under formal ABX test conditions (as shown later in the same article). Clearly, this suggests that ABX tests are in fact highly revealing of non-imaginary differences.
I think it depends on the type of difference. The brain doesn't analyze them all the same way. As I think I pointed out in another post, frequency response aberrations are analyzed almost immediately by what appears to be highly specialized neural circuitry. The same is likely true of timbre, which would include of course the detection of partials and harmonic distortion. What goes into long-term memory is for most of us an analyzed and simplified version of results. This however is useful when the phenomenon is too complex to analyze in real time.
Let me give you a real world example of that, involving vision, rather than sound. A few days ago, I repainted my bedroom. A previous sloppy paint job had left a ragged paint border on the edges of the wooden floor. After I pulled up the dropcloths and paint, I immediately noticed a new spot of paint, no more than a few millimeters wide. Sure enough, when I probed it it turned out to be a small fleck of paint that had fallen off the dropcloth. I would not have been able to make that determination in a real time A/B test, not without nauseatingly painstaking repetition, anyway. But the variations in the border had been recorded in my long-term memory, so I saw the difference immediately.
I think there is potentially an opposite extreme, as well -- steady-state distortions for which the brain compensates so rapidly that accommodation could occur even within the span of an ABX test.
It's not so much that I'm faulting ABX, as that I'm faulting any test that suffers from a particular constraint, sighted or unsighted. One of those is, as you say, the selection of source material. In practice, I find that this is particularly troubling with some resonant phenomena. It's no secret for example that some ribbons have serious distortion spikes at certain frequencies.
But then you have to fault every possible scientific test, don't you? They all suffer from constraints, every single one of them.
Of course that's why scientists repeat tests many times and refuse to accept them until other scientists have repeated them with the same results.
And even they they never report anything as an absolute truth but say things like "The probability of X lying somewhere between Y and Z is 99%", or if they don't then they aren't being scientists when that happens.
Ed
Sure. One of the things you learn pretty quickly in science is to take experimental results with the proverbial grain of salt. I have in my lifetime witnessed the discover of magnetic monopoles and cold fusion, not to mention any number of other phenomena that somehow vanished when others analyzed or attempted to reproduce the experiment. As Arnold said here, all scientific results are provisional, something that the popular press frequently doesn't understand.
Another thing you learn to recognize is the beautiful result -- the result that is compelling because it is astronomically impossible unless a theory conforms in some basic respect to the truth. And the elegant theory. It's surprising how often one can tell that a theory is wrong right out of the gate, or wonderfully right.
I repainted my bedroom.
Once you know where to look for that spot of paint, ABXing becomes trivial. No one has suggested that it is not OK for people to form subjective opinions, just that they should be verified though objective means.
Thank you for your anecdote. Keep trying.
Let me give you a real world example of that, involving vision, rather than sound. A few days ago, I repainted my bedroom. A previous sloppy paint job had left a ragged paint border on the edges of the wooden floor. After I pulled up the dropcloths and paint, I immediately noticed a new spot of paint, no more than a few millimeters wide. Sure enough, when I probed it it turned out to be a small fleck of paint that had fallen off the dropcloth. I would not have been able to make that determination in a real time A/B test, not without nauseatingly painstaking repetition, anyway. But the variations in the border had been recorded in my long-term memory, so I saw the difference immediately.
That would be analogous to ABX'ing a lossy encode against a wave file that you have listened to for years. That might make you more sensitive to any artifacts but a positive result still means you heard them and the blind test still works.
Once you know where to look for that spot of paint, ABXing becomes trivial. No one has suggested that it is not OK for people to form subjective opinions, just that they should be verified though objective means.
Thank you for your anecdote. Keep trying.
Weren't you the one who made that point already? Someone did. Anyway, I agreed: an ABX test conducted after long term listening could be a way of dealing with this potential inaccuracy. But I was answering Specific Impulses's question, "What makes you think that long-term listening is at all revealing of differences." The next question is, what kind of experiment could demonstrate or refute that assertion, and have any such experiments been conducted. Other than young Mozart's listening session at the Vatican, I mean.
But I was answering Specific Impulses's question, "What makes you think that long-term listening is at all revealing of differences."
Which is trivial and not particularly useful or interesting if someone is trying to argue that ABX is flawed because some artifacts can only be revealed through long term exposure. From what I've experienced, a common "artifact" being offered by placebophiles is listening fatigue.
But I was answering Specific Impulses's question, "What makes you think that long-term listening is at all revealing of differences."
Which is trivial and not particularly useful or interesting if someone is trying to argue that ABX is flawed because some artifacts can only be revealed through long term exposure. From what I've experienced, a common "artifact" being offered by placebophiles is listening fatigue.
Listening fatigue is, I think, another issue. Are you aware of any experiments that seek to quantify that, or the putative benefits of long-term listening? Because in their absence, I don't think there's much of an argument against these two assertions. At first glance, both types of experiment seem to me difficult to design, since statistically significant variations could potentially be attributed to multiple factors.
If someone is going to claim fatigue as a difference, the burden falls on him to demonstrate it.
The ABX skeptics throw up a lot of theories. They are nowhere near as prolific when it comes to providing evidence.
If someone is going to claim fatigue as a difference, the burden falls on him to demonstrate it.
An AES paper authored by Soren Bech that resulted from research connected with the 1980s Eureka Project demonstrated that listener fatigue in listening tests was an interfering variable. IIRC, his conclusion was that a blind test should be limited to less than one hour, to minimize the effect of the variable. I don't have the reference available right now, but I can post it if requested.
John Atkinson
Editor, Stereophile
An AES paper authored by Soren Bech that resulted from research connected with the 1980s Eureka Project demonstrated that listener fatigue in listening tests was an interfering variable. IIRC, his conclusion was that a blind test should be limited to less than one hour, to minimize the effect of the variable.
I don't see the relevance with regard to this discussion.
Let a subject S claim that item A sounds better than B. Common sense dictates that for such a claim S must at least be able to distinguish A and B. S can easily demonstrate this by double blindly identifying A and B. Using short time spans to not make it unnecessarily bothersome for him is fine (good practice, no objections).
But what if S claims, that an audible difference is there, but at the same time is unable to identify A and B or even a difference between both (mapping to X) under the latter conditions. At this point it is already likely that S is just pig-headed (unable/unwilling to modify a belief even if surrounded by facts).
But give S another chance to explain why an audible difference should still be assumed, although he could not identify it. S claims that the difference is only audible during prolonged listening. Grant him that in a second, unconstrained double blind installment. If he fails again, a sane conclusion is: there just was no audible difference, S is just fishing for reasons to retroactively justify his mental immobility. The other option is a circular fallacy: there is a difference, but S cannot identify it short-term (because it is only exposed long-term) and S cannot identify it long-term (because long-term testing is too fatiguing).
Using short time spans to not make it unnecessarily bothersome for him is fine (good practice, no objections).
By short timespans, you refer to the sample length, and not the length of a single session?
Because I'd think that doing many short AB[X] sessions would both satisfy the "long-term" requirement that S wants, as well as compensate for listener fatigue.
My point is that when a subjects claims a difference is only heard in the long-term (ie after becoming very familiar with the material), listener fatigue is not a valid complaint, as the ABX sessions can easily be broken down and spread out across days, even weeks. In fact, that would also compensate for tons of other hard to control environmental aspects, including the subject's mood, his ear wax or whether he got a good nights' sleep.
Yes, I agree.
If someone is going to claim fatigue as a difference, the burden falls on him to demonstrate it.
An AES paper authored by Soren Bech that resulted from research connected with the 1980s Eureka Project demonstrated that listener fatigue in listening tests was an interfering variable. IIRC, his conclusion was that a blind test should be limited to less than one hour, to minimize the effect of the variable. I don't have the reference available right now, but I can post it if requested.
The paper in question appears to be online and freely available:
Soren Bech AES paper URL (http://www.acourate.com/Download/BiasesInModernAudioQualityListeningTests.pdf)
Table 1 is particularly interesting. Means of bias reduction that it lists include:
"Use Blind Listening Tests"
and
"Use short, looped recordings with consistent characteristics"
The word *fatigue* does not appear in the body of the paper. I appears just //in the title of a work in the footnotes. I see no examples of common synonyms for fatigue, either.
It appears that Mr. Atkinson would do well to actually read this paper and take it to heart, as it contains a number of excellent criticisms of how Stereophile does its listening tests.
If someone is going to claim fatigue as a difference, the burden falls on him to demonstrate it.
An AES paper authored by Soren Bech that resulted from research connected with the 1980s Eureka Project demonstrated that listener fatigue in listening tests was an interfering variable. IIRC, his conclusion was that a blind test should be limited to less than one hour, to minimize the effect of the variable. I don't have the reference available right now, but I can post it if requested.
The paper in question appears to be online and freely available:
Soren Bech AES paper URL (http://www.acourate.com/Download/BiasesInModernAudioQualityListeningTests.pdf)
I was referring to this paper, reprinted in the July 1992 issue of the Journal of the AES: http://www.aes.org/e-lib/browse.cfm?elib=7040 (http://www.aes.org/e-lib/browse.cfm?elib=7040) . I am at work right now and this issue is at home, so I will check the reference this evening.
John Atkinson
Editor, Stereophile
Because I'd think that doing many short AB[X] sessions would both satisfy the "long-term" requirement that S wants, as well as compensate for listener fatigue.
My point is that when a subjects claims a difference is only heard in the long-term (ie after becoming very familiar with the material), listener fatigue is not a valid complaint, as the ABX sessions can easily be broken down and spread out across days, even weeks. In fact, that would also compensate for tons of other hard to control environmental aspects, including the subject's mood, his ear wax or whether he got a good nights' sleep.
That was the point I was making to greynol. Consider a formal test to examine the audibility of a small but possibly audible effect, which may not be audible on all kinds of music. You need a very large number of trials to bring the power of statistical anlaysis to bear, both on the audibility or lack thereof, and the interdependence between the effect and the music program.
Let's say that 200 ABX trials would give you an answer to a desired level of statistical significance. If you were to continue the testing with a single subject until he had done all 200 trials in a single session, I can coinfidently predict that the results would be no different from chance. But you haven't got meaningful results. What you need to do is allow the subject to do, say, 20 trials, then rest before doing another 20 trials, and so on. Because the time of day may well also be an interfering variable, it might also be best to allow the subject to do each set of 20 trials on consecutive days at the same time. This, for example, is how blind testing is done at B&O's research lab, which Tom Nousaine and I visited last spring.
John Atkinson
Editor, Stereophile
If someone is going to claim fatigue as a difference, the burden falls on him to demonstrate it.
An AES paper authored by Soren Bech that resulted from research connected with the 1980s Eureka Project demonstrated that listener fatigue in listening tests was an interfering variable. IIRC, his conclusion was that a blind test should be limited to less than one hour, to minimize the effect of the variable. I don't have the reference available right now, but I can post it if requested.
The paper in question appears to be online and freely available:
Soren Bech AES paper URL (http://www.acourate.com/Download/BiasesInModernAudioQualityListeningTests.pdf)
I was referring to this paper, reprinted in the July 1992 issue of the Journal of the AES: http://www.aes.org/e-lib/browse.cfm?elib=7040 (http://www.aes.org/e-lib/browse.cfm?elib=7040) . This issue is at home, so I will check the reference this evening.
I just checked the contents of this paper, and find a similar lack of mention of the word fatigue or common synonyms.
The two parameters that were studied in the referenced paper were related to the results of standard hearing tests and listener training by means of participationin the experiment.
The paper concluded:
"The results show that there is no correlation between the mean hearing threshold level and the mean standard deviation of the ratings for a group of subjects which
are screened to ensure that their hearing threshold level does not exceed 15 dB re ISO 389 [4] from 250 to 8000 Hz.
Note that the test subjects were between the ages of 18 and 28 years old.
"The effects of training are also discussed. The results suggest that 65% of the subjects will reach an asymptotic performance as measured by the magnitude of the error variance and the loudspeaker test statistic after four experiments. The result§ further indicate that the remaining 35% of the subjects will require a total of seven to eight training experiments before they stabilize in performance."
I think it depends on the type of difference. The brain doesn't analyze them all the same way.
Let's call the above what it is: It is unfounded speculation that there is an elusive kind of distortion that can be heard in long term tests and can't be heard in short term tests.
That is all it is - unfounded speculation with AFAIK zero reliable real world evidence to back it up.
As I think I pointed out in another post, frequency response aberrations are analyzed almost immediately by what appears to be highly specialized neural circuitry.
I notice that you are ignoring the fact that counter-evidence already cited was *not* based on frequency response AKA linear distortion, It was based on nonlinear distortion. Whether you know it or not, there are only two known kinds of distortion in this particular universe which are linear distortion and nonlinear distortion. Other relevant facts are that both linear and nonlinear distortion change the spectral balance of sounds and the ear is primarily a spectrum analyzer.
What goes into long-term memory is for most of us an analyzed and simplified version of results.
Common sense would take the above as a good explanation of why long term listening is so demonstrably insensitive to small differences.
AFAIK the only thing that long term listening is good for is finding those small portions of real-world recordings that reliably elicit the perception of an audible difference. The literature of good listening tests repeatedly shows that once those small segments are found, short-term listening to small snippets is the most effective way to demonstrate the presence of a difference.
That was the point I was making to greynol. Consider a formal test to examine the audibility of a small but possibly audible effect, which may not be audible on all kinds of music.
It is almost a given that small differences are only audible on a very small percentage of all music. In some cases the number of recordings where the difference is audible is a tiny fraction of all recordings and the portion of recordings where it is audible is only a tiny percentage of those few recordings.
You need a very large number of trials to bring the power of statistical analysis to bear, both on the audibility or lack thereof, and the interdependence between the effect and the music program.
No. Once you have formed your short list of candidate relevant recordings, you need to do more listening to just them to find the few most relevant segments. This can be done primarily using sighted evaluations. Blind tests need only be done in the final qualification stage. Only a small number of trials need to be done for each candidate selection. You usually only need a few final selections to run your actual test.
The other thing is that perchance you actually find something that takes say 50 trials per individual to obtain statistical significance, the listeners invariably report that they think that you are investigating something that makes no practical difference even though they obtained positive results. In many cases the listeners have no perception that they obtained statistically significant results. They thought they were just guessing.
That was the point I was making to greynol. Consider a formal test to examine the audibility of a small but possibly audible effect, which may not be audible on all kinds of music. You need a very large number of trials to bring the power of statistical anlaysis to bear, both on the audibility or lack thereof, and the interdependence between the effect and the music program.
How convenient. People claim to be fatigued by sighted listening to otherwise transparent lossy or CDDA but not by sighted listening to a high resolution counterpart, but they cannot be tested on this because unsighted listening is fatiguing???
EDIT: I fixed my quote. It made no impact on my statement, however.
I think it depends on the type of difference. The brain doesn't analyze them all the same way.
Let's call the above what it is: It is unfounded speculation that there is an elusive kind of distortion that can be heard in long term tests and can't be heard in short term tests.
That is all it is - unfounded speculation with AFAIK zero reliable real world evidence to back it up.
That's not what I said above. I said that the brain doesn't analyze all auditory differences in the same way. There is abundant, even overwhelming evidence for this, beginning with the fact that comb filtering is often interpreted by the brain as spatial consequences of reflections or the HRTF rather than as response anomalies, even when the anomalies are of a magnitude that surpasses the known thresholds for audibility. It does this processing almost in real time. You can see some of the anomalies *while* you move your head; once your head is stationary, the brain quickly compensates for the response differences.
I made no reference to some kind of mystery distortion, nor do I posit one.
As I think I pointed out in another post, frequency response aberrations are analyzed almost immediately by what appears to be highly specialized neural circuitry.
I notice that you are ignoring the fact that counter-evidence already cited was *not* based on frequency response AKA linear distortion, It was based on nonlinear distortion. Whether you know it or not, there are only two known kinds of distortion in this particular universe which are linear distortion and nonlinear distortion. Other relevant facts are that both linear and nonlinear distortion change the spectral balance of sounds and the ear is primarily a spectrum analyzer.
You have stated my own assumptions. I had thought them so obvious as to be trivial.
What goes into long-term memory is for most of us an analyzed and simplified version of results.
Common sense would take the above as a good explanation of why long term listening is so demonstrably insensitive to small differences.
AFAIK the only thing that long term listening is good for is finding those small portions of real-world recordings that reliably elicit the perception of an audible difference. The literature of good listening tests repeatedly shows that once those small segments are found, short-term listening to small snippets is the most effective way to demonstrate the presence of a difference.
I think the key phrase here is "AFAIK." It is, I grant, a distinct possibility. After all, in identifying the snippets that elicit the perception of an audible difference, long term memory would seem to have done the analytical work that gives it utility. That could be true even if the analysis was performed by somebody else. To use an analogy, someone not familiar with my bedroom would have great difficulty finding the paint chip if he were presented with images of the entire border, but would rapidly identify it if comparing preselected closeups of the specific region.
I suspect that there are still some differences that would be more identifiable in long term tests, based on the fact that even short audio snippets put heavy demands on short term memory. But that must remain speculation; I haven't seen any objective evidence for it, and some subjective observations, while perhaps food for thought, would not be allowable here and certainly don't have scientific weight.
We are both speculating here, not necessarily a bad thing, since it lays the ground for further research.
but they cannot be tested on this because unsighted listening is fatiguing???
You're sceptical about the mental effort required to give one's full attention to a complete ABX test?
I can't quantify it, though. When I want to have a go, I use foobar's ABX'er, but commonly tend to quit after a single incomplete run because I almost never hear a difference and my attention span doesn't last very long when I suspect the futility of my efforts.
Othe people may have greater discipline to complete several sittings.
The other thing is that perchance you actually find something that takes say 50 trials per individual to obtain statistical significance, the listeners invariably report that they think that you are investigating something that makes no practical difference even though they obtained positive results. In many cases the listeners have no perception that they obtained statistically significant results. They thought they were just guessing.
That, I think, is one of the most interesting facts in this thread.
I wonder to what extent such differences, which are apparently analyzed by some part of the brain but apparently not others, affect our experience of listening to music. It's akin to what I was wondering about 320 kbit/s MP-3's. As I said earlier, never having done any formal comparisons, I've never noticed any difference between high bit rate MP-3's and the uncompressed source material. But assuming that my ears aren't broken and I could be trained to identify them, would that training then be the sole determinant of my listening experience, or are the differences that I don't notice consciously influencing and perhaps diminishing my subjective experience, because perceived on a lower level?
That was the point I was making to greynol. Consider a formal test to examine the audibility of a small but possibly audible effect, which may not be audible on all kinds of music. You need a very large number of trials to bring the power of statistical anlaysis to bear, both on the audibility or lack thereof, and the interdependence between the effect and the music program.
How convenient. People claim to be fatigued by sighted listening to otherwise transparent lossy or CDDA but not by sighted listening to a high resolution counterpart, but they cannot be tested on this because unsighted listening is fatiguing???
As I said in the message of mine that you deleted, I have not said that. Nor does it follow from the message of mine from which you quoted. I was making a general argument concerning the possibility of listener fatigue acting as an interfering variable in formal listening tests and what can be done to prevent that from occurring.
John Atkinson
Editor, Stereophile
I wonder to what extent such differences, which are apparently analyzed by some part of the brain but apparently not others, affect our experience of listening to music. It's akin to what I was wondering about 320 kbit/s MP-3's. As I said earlier, never having done any formal comparisons, I've never noticed any difference between high bit rate MP-3's and the uncompressed source material. But assuming that my ears aren't broken and I could be trained to identify them, would that training then be the sole determinant of my listening experience, or are the differences that I don't notice consciously influencing and perhaps diminishing my subjective experience, because perceived on a lower level?
Reality is that differences that you can ABX in a heartbeat often have zero effect on your listening pleasure.
For example, ABX the same musical selection, one unchanged, the other attenuated by a dB or two. In open listening both are equally enjoyable. Yet you can easily ABX them and hear a difference.
Just enough nonlinear distortion to be easily ABX-able can have similar effects, even high orer distortion. As I pointed out in a previous post, one of the subjective effects of modest amounts of common forms of nonlinear distortion might be a slight shift in timbre.
While we call common forms of nonlinear distortion "grunge", it doesn't sound like grunge in modest amounts - enough to be just barely reliably detectable. You hear it as a barely discernable difference that you can't detect at all without a ready undistorted referemce. Listen to it all day, and you'll never know that it is there. The things that actually detract from your listening pleasure are pretty gross.
As I said in the message of mine that you deleted, I have not said that. Nor does it follow from the message of mine from which you quoted. I was making a general argument concerning the possibility of listener fatigue acting as an interfering variable in formal listening tests and what can be done to prevent that from occurring.
You didn't suggest that ABX testing causes fatigue? If you did then your complaints are a non sequitur. My question was not posed to you personally. If it were I would have addressed you personally. This is a public forum, John; my comments are open for anyone to respond.
The things that actually detract from your listening pleasure are pretty gross.
Like sighted listening to non-high resolution lossless? I truly believe that some are detracted from their listening pleasure knowing that all the "sonic goodness" hasn't been exploited. What about the poor fool who realized his magic rocks were stolen? Do you think his system will sound the same to him in his conscious mind when he knows they aren't there?
I suspect that there are still some differences that would be more identifiable in long term tests,
Looks like a statement of religious faith.
based on the fact that even short audio snippets put heavy demands on short term memory.
No, they reduce the demands on short term memory and eliminate demands on long term memory.
We are both speculating here, not necessarily a bad thing, since it lays the ground for further research.
No, I'm not speculating at all, except about the tiny details of very specific cases. In general, I know what to suspect,, based on decades of practical experience. I can't say that my experience covers 100.000% of every situation, but I've formed general rules and guidelines that predict the outcome of almost all listening tests.
If I make mistakes at all, I usually end up failing to predict null outcomes. For example when we recently did blind comparisons of some <$400 a pair powered monitors to some highly regarded $12,000 a pair systems I didn't expect that the listeners would have no significant preferences for one over the other. Hope springs eternal!
The things that actually detract from your listening pleasure are pretty gross.
Like sighted listening to non-high resolution lossless? I truly believe that some are detracted from their listening pleasure knowing that all the "sonic goodness" hasn't been exploited.
Obviously true for some of audio's more legendary obsessive-compulsives like Mr. F.
What about the poor fool who realized his magic rocks were stolen? Do you think his system will sound the same to him in his conscious mind when he knows they aren't there?
I think the way it works is that people hear of some gizmo that will *improve* their sound based on hearty testimonials from the usual suspects, and then their system's sound causes them unbearable stress until they *fix* the situation by means of a new equipment acquisition. That's how I was until I had my eyes opened by ABX.
That was the point I was making to greynol. Consider a formal test to examine the audibility of a small but possibly audible effect, which may not be audible on all kinds of music. You need a very large number of trials to bring the power of statistical anlaysis to bear, both on the audibility or lack thereof, and the interdependence between the effect and the music program.
How convenient. People claim to be fatigued by sighted listening to otherwise transparent lossy or CDDA but not by sighted listening to a high resolution counterpart, but they cannot be tested on this because unsighted listening is fatiguing???
As I said in the message of mine that you deleted, I have not said that. Nor does it follow from the message of mine from which you quoted. I was making a general argument concerning the possibility of listener fatigue acting as an interfering variable in formal listening tests and what can be done to prevent that from occurring.
You didn't suggest that ABX testing causes fatigue? If you did then your complaints are a non sequitur.
I thought it was clear in my comment that I was talking about fatigue in general. The test subjects in ABX tests become fatigued, just as they do in any task requiring concentration. This isn't a comment on the ABX methodology as such; my point is that a well-designed listening test needs to allow for the inevitable fatigue if it is not to become an interfering variable. This is hardly controversial.
My question was not posed to you personally. If it were I would have addressed you personally. This is a public forum, John; my comments are open for anyone to respond.
Of course, It was just that as you were quoting from one of my postings, that implied that you were commenting on something I was supposed to have written. If that were not the case, then my apologies.
John Atkinson
Editor, Stereophile
I think the way it works is that people hear of some gizmo that will *improve* their sound based on hearty testimonials from the usual suspects, and then their system's sound causes them unbearable stress until they *fix* the situation by means of a new equipment acquisition.
Right. Discovering the disappearance of the magic rocks will cause the believer to revert back to his previous state of unbearable stress. When the believer realizes that the rocks could have been gone for listening sessions prior to the discovery, he will re-interpret his experiences from these prior sessions in order to satisfy his belief.
Reality is that differences that you can ABX in a heartbeat often have zero effect on your listening pleasure.
For example, ABX the same musical selection, one unchanged, the other attenuated by a dB or two. In open listening both are equally enjoyable. Yet you can easily ABX them and hear a difference.
Just enough nonlinear distortion to be easily ABX-able can have similar effects, even high orer distortion. As I pointed out in a previous post, one of the subjective effects of modest amounts of common forms of nonlinear distortion might be a slight shift in timbre.
While we call common forms of nonlinear distortion "grunge", it doesn't sound like grunge in modest amounts - enough to be just barely reliably detectable. You hear it as a barely discernable difference that you can't detect at all without a ready undistorted referemce. Listen to it all day, and you'll never know that it is there. The things that actually detract from your listening pleasure are pretty gross.
An interesting point. I wonder, though, whether even if it isn't offensive, it doesn't detract from the sense of realism, which is in itself pleasurable. But I can't think of any objective tests from which to quote here, as I could in the case of e.g. frequency response aberrations that have been linked inversely to listener assessments of subjective quality.
I suspect that there are still some differences that would be more identifiable in long term tests,
Looks like a statement of religious faith.
Nope, it looks like a statement of agnosticism. Religious faith would be taking either position in the absence of evidence.
based on the fact that even short audio snippets put heavy demands on short term memory.
No, they reduce the demands on short term memory and eliminate demands on long term memory.
Reread what I wrote.
We are both speculating here, not necessarily a bad thing, since it lays the ground for further research.
No, I'm not speculating at all, except about the tiny details of very specific cases. In general, I know what to suspect,, based on decades of practical experience. I can't say that my experience covers 100.000% of every situation, but I've formed general rules and guidelines that predict the outcome of almost all listening tests.
If I make mistakes at all, I usually end up failing to predict null outcomes. For example when we recently did blind comparisons of some <$400 a pair powered monitors to some highly regarded $12,000 a pair systems I didn't expect that the listeners would have no significant preferences for one over the other. Hope springs eternal!
If you aren't speculating, you haven't presented your complete case here, because I haven't seen anything that nullifies the postulated objections some have to ABX testing. I have seen what, on the face of it, would seem to be useful techniques to minimize some of those objections, e.g., preselecting program material and, perhaps, educating the ear through long-term listening before performing the tests. Taken together, those techniques would seem to dispense with some major objections.
I suspect that there are still some differences that would be more identifiable in long term tests,
Looks like a statement of religious faith.
Nope, it looks like a statement of agnosticism. Religious faith would be taking either position in the absence of evidence.
An agnostic statement would be to say that there are some differences that are equally identifiable either way. Thus far we know that short term tests are generally superior.
We are both speculating here, not necessarily a bad thing, since it lays the ground for further research.
No, I'm not speculating at all, except about the tiny details of very specific cases. In general, I know what to suspect,, based on decades of practical experience. I can't say that my experience covers 100.000% of every situation, but I've formed general rules and guidelines that predict the outcome of almost all listening tests.
If I make mistakes at all, I usually end up failing to predict null outcomes. For example when we recently did blind comparisons of some <$400 a pair powered monitors to some highly regarded $12,000 a pair systems I didn't expect that the listeners would have no significant preferences for one over the other. Hope springs eternal!
If you aren't speculating, you haven't presented your complete case here,
I've presented a quick summary. I'm not going to post a full academic paper for formal review here.
because I haven't seen anything that nullifies the postulated objections some have to ABX testing.
That's anything but agnosticism. You've just thrown in with "..the postulated objections that some have to ABX testing."
Besides, your criteria is set way to high and is biased against ABX. There are few theories that are totally nullified in the eyes of diehards who hold onto them in the face of considerable counter evidence. We don't have to nullify tho theories of nay-sayers, all we have to do is advance theories and evidence that establish that the preponderance of valid evidence supports ABX. That was done decades ago.
If you simply ask that listening tests be valid tests, then sighted evaluations die on the spot.
I have seen what, on the face of it, would seem to be useful techniques to minimize some of those objections, e.g., preselecting program material and, perhaps, educating the ear through long-term listening before performing the tests. Taken together, those techniques would seem to dispense with some major objections.
"Educating the ear via long term tests" fails on the ground that in every case we've examined, long term tests are woefully and unnecessarily time-consuming and insensitive. I see that you apparently never bothered to read "Flying Blind", or you didn't get it.
An agnostic statement would be to say that there are some differences that are equally identifiable either way. Thus far we know that short term tests are generally superior.
I said nothing contradicting, or affirming, that, since I'm not aware of any hard evidence that they generally are or aren't, only evidence that they are superior in some cases.
because I haven't seen anything that nullifies the postulated objections some have to ABX testing.
That's anything but agnosticism. You've just thrown in with "..the postulated objections that some have to ABX testing."
No, that's why I referred to them as "postulated." For the most part, I'm not aware of any solid evidence that they're valid. In some cases where there is evidence, e.g., of the statistical difficulty of detecting low probability events in ABX tests of practical length, you've pointed to techniques that might reduce the problem.
You seem to be trying to place me in a camp here. That will generally fail, because even where I do have a personal agenda, e.g., a distaste for snake oil, I try to keep it separate from discussions of the science. Wishing that I could prove something won't make it so. The best I can do is try to narrow any areas of uncertainty.
Besides, your criteria is set way to high and is biased against ABX. There are few theories that are totally nullified in the eyes of diehards who hold onto them in the face of considerable counter evidence. We don't have to nullify tho theories of nay-sayers, all we have to do is advance theories and evidence that establish that the preponderance of valid evidence supports ABX. That was done decades ago.
If you simply ask that listening tests be valid tests, then sighted evaluations die on the spot.
I'm raising objections here rather than trying to balance evidence. That's just a matter of the direction in which the discussion ran. If you step over to the critic's asylum, you'll see that I've most recently been debating the audio applications of quantum teleportation with Geoff Kait -- great fun, more for the opportunity to discuss quantum mechanics than any practical point about audio -- and defending Nyquist. And Jim Austin.
I have seen what, on the face of it, would seem to be useful techniques to minimize some of those objections, e.g., preselecting program material and, perhaps, educating the ear through long-term listening before performing the tests. Taken together, those techniques would seem to dispense with some major objections.
"Educating the ear via long term tests" fails on the ground that in every case we've examined, long term tests are woefully and unnecessarily time-consuming and insensitive. I see that you apparently never bothered to read "Flying Blind", or you didn't get it.
I'd forgotten about "Flying Blind," I must have gotten sidetracked.
You yourself have spoken of the benefits of listening to material at sufficient length to identify sections that are candidates for ABX tests of audible phenomena with a low probability of occurrence.
It is also, I think, well established that training increases the ability of listeners to make consistent judgments; see Olive and Toole.
As for not getting things, I don't get women, I don't get Edith Piaf, and I don't get people who don't get Keynesian economics.
An agnostic statement would be to say that there are some differences that are equally identifiable either way. Thus far we know that short term tests are generally superior.
I'm not aware of any hard evidence that they generally are or aren't, only evidence that they are superior in some cases.
I'd forgotten about "Flying Blind," I must have gotten sidetracked.
Please come back when you've done your homework. You can't possibly have a well-informed opinion that is worth the time to address as long as you ignore evidence that you have been pointed again and again at for days, and conveniently forget to look at.
Here's a link to the article PDF, and some other articles you probably need to read and understand as well:
Links to "Flying Blind" and other relevant Tom Nousaine articles (http://nousaine.com/nousaine_tech_articles.html)
An agnostic statement would be to say that there are some differences that are equally identifiable either way. Thus far we know that short term tests are generally superior.
I'm not aware of any hard evidence that they generally are or aren't, only evidence that they are superior in some cases.
I'd forgotten about "Flying Blind," I must have gotten sidetracked.
Please come back when you've done your homework. You can't possibly have a well-informed opinion that is worth the time to address as long as you ignore evidence that you have been pointed again and again at for days, and conveniently forget to look at.
Here's a link to the article PDF, and some other articles you probably need to read and understand as well:
Links to "Flying Blind" and other relevant Tom Nousaine articles (http://nousaine.com/nousaine_tech_articles.html)
All right, I read the article. It was a very nice article, really it was, and worth reading, even though you'd already described the results here.
Perhaps, in time, I will look at some of the others, but really, even my dog knows that some kinds of distortion are most audible in short term comparisons. That has never been a matter of debate. The question is whether some are best heard with long-term listening.
I have stipulated that some forms of distortion are most audible in short term comparisons, and you have already stipulated that some forms of distortion benefit from pre-selection of program material, which requires long term listening. I have also expressed the opinion that the methodology you propose -- long-term listening followed by short-term comparisons -- is probably sufficient to overcome the problem of events with a low probability of occurrence, as well as to allow any necessary training to occur. This does not dispense with all objections, but insofar as these issues are concerned, with the exception perhaps of the utility of training by listening and the utility of ad hominem attacks and posturing, I do not see that we have any fundamental disagreement.
The question is whether some are best heard with long-term listening.
Nobody has ever been able to name that distortion or provide an example of it.
I have stipulated that some forms of distortion are most audible in short term comparisons,
A meaningless *concession*. Some = what, 2? Who knows?
you have already stipulated that some forms of distortion benefit from pre-selection of program material, which requires long term listening.
No, I've stipulated that the nature of defects in recordings which includes distortion is such that not every second of every recording ever made is the best place to hear it.
That is a common sense finding that has nothing to do with long term versus short term listening.
I have also expressed the opinion that the methodology you propose -- long-term listening followed by short-term comparisons -- is probably sufficient to overcome the problem of events with a low probability of occurrence, as well as to allow any necessary training to occur.
There's no need for long term listening once someone someplace figured out which parts of which recordings are good for hearing whatever you want to hear.
This is no different from listening for pleasure in that you have to use similar means to listen to a piece of music you want to hear. You have to find the right part of the right disc to hear that piece of music you want to hear.
This does not dispense with all objections,
It only dispenses with the known reasonable objections.
but insofar as these issues are concerned, with the exception perhaps of the utility of training by listening and the utility of ad hominem attacks and posturing, I do not see that we have any fundamental disagreement.
You made yourself an easy target for criticism when you continued to pretend that there is no evidence after several people pointed you at it. Well with some significant effort we finally worked you past that...
The question is whether some are best heard with long-term listening.
Nobody has ever been able to name that distortion or provide an example of it.
I have given several examples of such distortion, to whit, distortions that are highly dependent on the program material. You have yourself acknowledged this phenomenon. It is an inevitable consequence of trying to conduct an experiment with a test signal with low self-correlation.
I have stipulated that some forms of distortion are most audible in short term comparisons,
A meaningless *concession*. Some = what, 2? Who knows?
It is not a concession, it as an age-old observation, and neither is it meaningless. If it were meaningless, there would be no reason to conduct short term listening comparisons other than convenience. There clearly is. And I believe many, if not all, subjective reviewers acknowledge that.
you have already stipulated that some forms of distortion benefit from pre-selection of program material, which requires long term listening.
No, I've stipulated that the nature of defects in recordings which includes distortion is such that not every second of every recording ever made is the best place to hear it.
That is a common sense finding that has nothing to do with long term versus short term listening.
"You have stipulated that some houses are red." "No, I have stipulated that some houses are red."
I fail to see how this can have nothing to do with long versus short term listening. To identify some audible distortions requires listening to a wide variety of program material. This is, I think, almost self-evident: a piccolo suite doesn't tell us much about the behavior of a woofer.
You made yourself an easy target for criticism when you continued to pretend that there is no evidence after several people pointed you at it. Well with some significant effort we finally worked you past that...
Save it for the jury. I don't pretend, I have acknowledged all evidence presented here, and you delude yourself if you suppose that anyone whose opinion I care about is swayed by bluster or pomposity. as opposed to technical arguments and facts.
Josh:
If you aren't speculating, you haven't presented your complete case here, because I haven't seen anything that nullifies the postulated objections some have to ABX testing. I have seen what, on the face of it, would seem to be useful techniques to minimize some of those objections, e.g., preselecting program material and, perhaps, educating the ear through long-term listening before performing the tests. Taken together, those techniques would seem to dispense with some major objections.
Sure. Academic ABX/DBT testing typically includes training, and protocols taht take fatigue into account.
But maybe first the 'objectors' should do some objecting to the patently nonrigorous means of determining audible difference employed by the loudest and most attended-to voices in audiophilia -- voices like
Stereophile and
TAS? Indeed, some of the 'objectors' are writers and editorial staff for those magazines.
For someone like John Atkinson to publish 'objections' to ABX while touting difference-signal 'demonstrations' of the 'evils of MP3' strikes me as....objectionable.
Ive been reading the arguments going on here and thought this topic on ABX testing is very interesting especially because Ive been doing these abx test with foobar myself and have also been a stereophile reader since the early 90s and actually also respect what John Atkinson has to say on stereophile since Ive experienced a lot of his findings on certain things being similar to mine. You guys could say i'm in the neutral zone here because I find value in all sides of the arguments.
The reason why I felt like posting here was not because I wanted to argue with anyone or talk about the specific audible differences I hear with the different lossy file formats because the flowery words I might use to describe the differences I hear is sure to be a TOS8 violation. Unless I can state exactly what the technical term for the deficiencies that I hear are ex. mp3's pre-echo in certain passages and other specific errors on different passages then I'd rather not talk about anything specific on this topic. These sound cues/deficiencies that I hear are what make me able to pass abx tests on foobar even though I don't know what type of errors they are.
Before losing myself in writing too many things here, I just have some questions on the top of my head that would be interesting if anyone could answer with scientific proof here. My questions are...
-Is it proven that not being able to pass ABX tests on foobar comparing two files mean that they are sonically the same in quality? What if you don't exclusively listen to foobar/pc audio and you can hear the difference on your home audio rig more clearly but have no way to prove it because you can not use foobar abx during the time you are hearing those limitations?
- Does the lack of training for searching for specific codec artifacts when doing an abx test mean that one will not be able to hear those differences when just listening to the music for enjoyment from time to time? Its possible that one could be focused on too many specific things trying to find the differences between files when they should listen to other parts of the music instead during an abx test. Does this really mean that failing to pass a quick ABX test ensures that one cannot hear the difference between these files in the long term?
-Is it possible that in certain mental states (different moods, sleepiness, etc...) that certain differences in sound can be heard more easily by people? This one is a mystery that I cannot cite a specific example here that doesn't violate TOS8.
-and here is a funny question... Why is Placebo bad? Since we are all more or less educated enough here to know that lossless files are bigger than lossy files in filesize, why is placebo a bad thing if it makes a person enjoy listening to music via lossless files more than lossy just because one thinks deep down inside that there is less of a possibility of losses when listening to lossless?
-and here is a funny question... Why is Placebo bad?
Because there are real resources involved, like hard drive space, CPU, time, and, last but not least,
money.
Placebo isn't bad, but if placebo causes you to needlessly waste money, then that's bad.
-Is it proven that not being able to pass ABX tests on foobar comparing two files mean that they are sonically the same in quality?
No. It means one couldn't distinguish between the two files on that particular setup, on that particular time, etc. How far you want to extrapolate from that varies.
- Does the lack of training for searching for specific codec artifacts when doing an abx test mean that one will not be able to hear those differences when just listening to the music for enjoyment from time to time?
No, but the possibility to find differences when you are not looking for them and without having a reference doesn't strike me to be higher. On top of that an ABX test doesn't have to have a short time span. If you think you can reliably hear a difference later on, it is allowed to come back and ABX again.
-and here is a funny question... Why is Placebo bad? Since we are all more or less educated enough here to know that lossless files are bigger than lossy files in filesize, why is placebo a bad thing if it makes a person enjoy listening to music via lossless files more than lossy just because one thinks deep down inside that there is less of a possibility of losses when listening to lossless?
Because placebo often involves spending more money.
LOL I like those replies about money. Money is a very important thing not to waste but if one thinks about the cost of hard drive space these days though its probably less than a dime per gigabyte Is that really worth not squeezing out all the possible juice out of the $17 CD one just bought? Thats approximately three albums per dime that don't have to be ripped and re-ripped all over again when the newest builds of lossy codecs come along sounding better per byte than its predecessor.
LOL I like those replies about money. Money is a very important thing not to waste but if one thinks about the cost of hard drive space these days though its probably less than a dime per gigabyte
Give a look at the price tag of the equipments usually reviewed on audiophile magazines (Stereophile and the like)...
P.S. By the way, Mr. Atkinson, I'm a subscriber of yours... mainly for economical reasons: the iPhone only version is sooo cheap!
-and here is a funny question... Why is Placebo bad? Since we are all more or less educated enough here to know that lossless files are bigger than lossy files in filesize, why is placebo a bad thing if it makes a person enjoy listening to music via lossless files more than lossy just because one thinks deep down inside that there is less of a possibility of losses when listening to lossless?
IMO, because an untruth/misconception being comforting doesn’t negate the fact that it’s an untruth/misconception.
Is that really worth not squeezing out all the possible juice out of the $17 CD one just bought? Thats approximately three albums per dime that don't have to be ripped and re-ripped all over again when the newest builds of lossy codecs come along sounding better per byte than its predecessor.
This PDF (http://nousaine.com/pdfs/Wired%20Wisdom.pdf) (connection may be slow), on "page 76" has an inset called Table 2, and it embodies the core of what I'm talking about.
-and here is a funny question... Why is Placebo bad? Since we are all more or less educated enough here to know that lossless files are bigger than lossy files in filesize, why is placebo a bad thing if it makes a person enjoy listening to music via lossless files more than lossy just because one thinks deep down inside that there is less of a possibility of losses when listening to lossless?
This is probably the first question many people ask about placebo. Long time ago I asked it myself to a doctor friend of mine, and his answer was simple and can pretty much be applied in most or all instances (even outside of medicine, I mean). It's dishonest and unethical. With doctors, it can potentially lose patient trust. I don't think with pseudoscientists in general (audiophools, homeopaths, shamans, whatever) trust is an issue, since they feed in gullibility; they'll just find other targets if they lose trust with the previous ones. Of course, it also has a cost like others said (time and money in the case of audiophilia, but can be worse in other cases, it can even cost people's lives for instance in medical cases).
btw, there was a guy some time ago here that was asking for advice for his friend, whose marriage was suffering because of his extravagant spending in audiophile stuff.
-Is it proven that not being able to pass ABX tests on foobar comparing two files mean that they are sonically the same in quality?
-and here is a funny question... Why is Placebo bad? Since we are all more or less educated enough here to know that lossless files are bigger than lossy files in filesize, why is placebo a bad thing if it makes a person enjoy listening to music via lossless files more than lossy just because one thinks deep down inside that there is less of a possibility of losses when listening to lossless?
Because placebo often involves spending more money.
If some bored dilettantes spending their trust fund money on expensive placebo-audio systems was the extent of it, I'd be a lot happier.
But it isn't. The interesting question would be what percentage of all audio R&D money is being spent on things that are placebo-related. Another question would be how much effort is was
wasted on mainstream equipment to address audiophile myths. Then there is what I see as the now largely ruined high end audio market that at one time exclusively addressed real improvements in sound quality. It is possible that we'd now have far better actual audio technology if the floobydust contingent hadn't siphoned off so much of everybodies' time and energy.
It is possible that we'd now have far better actual audio technology if the floobydust contingent hadn't siphoned off so much of everybodies' time and energy.
...and relatively normal people would still be interested
You know, in the same way that relatively normal people think "HD looks better", but don't think "this $500 cable sounds better".
Cheers,
David.
For someone like John Atkinson to publish 'objections' to ABX while touting difference-signal 'demonstrations' of the 'evils of MP3' strikes me as....objectionable.
Why? This is a serious question. Like all human endeavors, the utility of ABX is open to debate. I have taken part in well over 100 formal blind tests: double-blind, single-blind using methodology approved, for example, by Stanley Lipshitz. I have even used the ABX Box. My criticisms are not fabricated out of whole cloth; I feel it appropriate to publish those criticisms in Stereophile, though I acknowledge that ti would probably not be appropriate to offer them on HA.
As for difference tests, I feel they can be useful, as do other engineers whose opinions I respect. Others, as has pointed out in this thread, disagree. Why do you have a problem with me personally holding these opinions and discussing them when and where I feel appropriate?
John Atkinson
Editor, Stereophile
Unless I can state exactly what the technical term for the deficiencies that I hear are ex. mp3's pre-echo in certain passages and other specific errors on different passages then I'd rather not talk about anything specific on this topic.
...or tell us you'll get back with more test results in a week (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=38881&view=findpost&p=344130) and never do?
What if you don't exclusively listen to foobar/pc audio and you can hear the difference on your home audio rig more clearly but have no way to prove it because you can not use foobar abx during the time you are hearing those limitations?
There are other methods of doing ABX than foobar, but using foobar with proper time-synchronized and level-matched samples this is very easily controlled. Why can't you connect your PC to your stereo?
- Does the lack of training for searching for specific codec artifacts when doing an abx test mean that one will not be able to hear those differences when just listening to the music for enjoyment from time to time?
Most likely, yes.
Its possible that one could be focused on too many specific things trying to find the differences between files when they should listen to other parts of the music instead during an abx test.
Doubtful.
Does this really mean that failing to pass a quick ABX test ensures that one cannot hear the difference between these files in the long term?
If you cannot distinguish the difference in the short term, then you probably will not be able to in the long term either, unless you've managed to successfully train yourself, in which case you can test again.
-Is it possible that in certain mental states (different moods, sleepiness, etc...) that certain differences in sound can be heard more easily by people?
...or your ears are stuffed up because of a cold, sure. As I said before, feel free to test again.
My criticisms are not fabricated out of whole cloth; I feel it appropriate to publish those criticisms in Stereophile, though I acknowledge that ti would probably not be appropriate to offer them on HA.
It's not inappropriate, so long as you follow teh rulez (http://www.hydrogenaudio.org/forums/index.php?showtopic=3974).
For someone like John Atkinson to publish 'objections' to ABX while touting difference-signal 'demonstrations' of the 'evils of MP3' strikes me as....objectionable.
Why? This is a serious question.
Maybe because you're not a casual contributor, but someone with "an agenda"? The agenda presumably being to make a magazine about very expensive audio equipment, sell.
As for difference tests, I feel they can be useful, as do other engineers whose opinions I respect.
I know - but some of these others really should know better. Others are saying "this is what the codec is
doing to the signal" and then making clear that the magnitude or nature of the difference you hear in a difference test is
not necessarily related to the magnitude or nature of difference you hear in an A/B comparison.
I notice the same people are less keen to demonstrate the difference signal in the case of 24-bits vs 16-bits, never mind 48kHz sampling vs 96kHz sampling.
(I really can't find a smiley big enough and cynical enough to put at the end of that sentence!)
Cheers,
David.
I notice the same people are less keen to demonstrate the difference signal in the case of 24-bits vs 16-bits, never mind 48kHz sampling vs 96kHz sampling.
For someone like John Atkinson to publish 'objections' to ABX while touting difference-signal 'demonstrations' of the 'evils of MP3' strikes me as....objectionable.
Why? This is a serious question.
Maybe because you're not a casual contributor, but someone with "an agenda"? The agenda presumably being to make a magazine about very expensive audio equipment, sell.
I think it fair to point out that that assumes matters that "not in evidence," as they say. But it would be fruitless to argue the matter. People believ what they want to believe and disregard the rest, to paraphrase Paul Simon.
As for difference tests, I feel they can be useful, as do other engineers whose opinions I respect.
I know - but some of these others really should know better. Others are saying "this is what the codec is doing to the signal" and then making clear that the magnitude or nature of the difference you hear in a difference test is not necessarily related to the magnitude or nature of difference you hear in an A/B comparison.
I made that exact point in the Seattle dems. But playing the difference signal emphasizes that _something_ is removed by lossy codecs, which is a point that I also wanted to make.
I notice the same people are less keen to demonstrate the difference signal in the case of 24-bits vs 16-bits, never mind 48kHz sampling vs 96kHz sampling.
I did demonstrate the former at the 2004 AES Convention in the workshop on high-resolution players. Some 24-bit recordings have nothin but random activity in the 7 or LSBs. Others do have real musical information down there.
John Atkinson
Editor, Stereophile
It is possible that we'd now have far better actual audio technology if the floobydust contingent hadn't siphoned off so much of everybodies' time and energy.
...and relatively normal people would still be interested
You know, in the same way that relatively normal people think "HD looks better", but don't think "this $500 cable sounds better".
Right - the same kind of people if not the same people who in 1983 decided that CDs in general sounded better than the LPs that they already had. The high end has been trying to tell them that they were wrong ever since.
-and here is a funny question... Why is Placebo bad?
Because there are real resources involved, like hard drive space, CPU, time, and, last but not least, money.
Placebo isn't bad, but if placebo causes you to needlessly waste money, then that's bad.
"Waste" here is as per your perspective. From the audiophile's perspective, pleasure is derived as money is spent.
In other news, the placebo effect is sometimes strong enough to overcome any ethical issues - http://www.cbc.ca/news/health/story/2010/1...bs.html?ref=rss (http://www.cbc.ca/news/health/story/2010/12/22/placebo-effect-ibs.html?ref=rss)
For someone like John Atkinson to publish 'objections' to ABX while touting difference-signal 'demonstrations' of the 'evils of MP3' strikes me as....objectionable.
Why? This is a serious question. Like all human endeavors, the utility of ABX is open to debate. I have taken part in well over 100 formal blind tests: double-blind, single-blind using methodology approved, for example, by Stanley Lipshitz. I have even used the ABX Box. My criticisms are not fabricated out of whole cloth; I feel it appropriate to publish those criticisms in Stereophile, though I acknowledge that ti would probably not be appropriate to offer them on HA.
'Publish' as in 'allow to be published in your magazine, in your role as editor' not just as in 'write yourself'.
As for difference tests, I feel they can be useful, as do other engineers whose opinions I respect. Others, as has pointed out in this thread, disagree. Why do you have a problem with me personally holding these opinions and discussing them when and where I feel appropriate?
John Atkinson
Editor, Stereophile
Your magazine has taken a rather consistently 'skeptical' stance towards audio DBTs, and of course your own writing is part of that too. Every theoretical objection to DBTs -- a cornerstone method in psychacoustic research --gets an airing at Stereophile. Meanwhile, objections to the use of the 'difference' method to demonstrate what's evil about mp3s seem a bit
rare on the page, in
Stereophile. This is the case even though it's a patently lousy and misleading method for demonstrating how mp3s are heard. Shall I find you some 'engineers' that might agree with me on that? I can think of one that you know and respect that even posts here.
Simpler version: established scientific method: ALWAYS QUESTIONED. easily discredited , pseudoscientific method: TAKEN ON THE ROAD.
Yeah, I object to that.
Others are saying "this is what the codec is doing to the signal" and then making clear that the magnitude or nature of the difference you hear in a difference test is not necessarily related to the magnitude or nature of difference you hear in an A/B comparison.
I made that exact point in the Seattle dems. But playing the difference signal emphasizes that _something_ is removed by lossy codecs, which is a point that I also wanted to make.
What, pray tell, is the point, other than 'ideological', in
emphasizing that something is
lost when a
lossy perceptual codec is used, particularly when the demonstrated 'loss' is not necessarily correlated to what the mp3 sounds like? What 'evil' was demonstrated? Perhaps you would consider emphasizing the *perceptual* part at your next dem. The listeners might be truly amazed to hear what the use of perceptual models can achieve in terms of codec transparency.
In other news, the placebo effect is sometimes strong enough to overcome any ethical issues - http://www.cbc.ca/news/health/story/2010/1...bs.html?ref=rss (http://www.cbc.ca/news/health/story/2010/12/22/placebo-effect-ibs.html?ref=rss)
There were several problems with that study, it wasn't exactly rock solid. One of the most glaring faults being:
Before randomization and during the screening, the placebo pills were truthfully described as inert or inactive pills, like sugar pills, without any medication in it. Additionally, patients were told that “placebo pills, something like sugar pills, have been shown in rigorous clinical testing to produce significant mind-body self-healing processes.”
IOW, they placeboed the placebos.
See here (http://www.sciencebasedmedicine.org/?p=9339) and here (http://blogs.discovermagazine.com/notrocketscience/2010/12/22/evidence-that-placebos-could-work-even-if-you-tell-people-they%E2%80%99re-taking-placebos/) for more details.
"Waste" here is as per your perspective. From the audiophile's perspective, pleasure is derived as money is spent.
That is true. I guess it just conflicts with my personal sense of worldly justice and cosmic balance.
I notice the same people are less keen to demonstrate the difference signal in the case of 24-bits vs 16-bits, never mind 48kHz sampling vs 96kHz sampling.
I did demonstrate the former at the 2004 AES Convention in the workshop on high-resolution players. Some 24-bit recordings have nothin but random activity in the 7 or LSBs. Others do have real musical information down there.
I'm not sure how you'd reliably tell the difference. I have an interest in how many bits of LPCM include useful information (http://wiki.hydrogenaudio.org/index.php?title=Lossywav) and it's not a trivial question to answer. If the last 7-bits of 24-bit audio were audible, then you increased the gain
a little . If they sounded like anything other than white noise, that's quite surprising, though not impossible in contrived signals.
We've got to be very careful here though: a 16-bit signal isn't
just a 24-bit signal with the bottom 8-bits chopped off. There should be dither - preferably noise shaped dither. Do that, and the 24-bit vs 16-bit difference signal is always dominated by the dither, and never contains anything recognisable from the original - that's the point of dither.
Cheers,
David.
Unless I can state exactly what the technical term for the deficiencies that I hear are ex. mp3's pre-echo in certain passages and other specific errors on different passages then I'd rather not talk about anything specific on this topic.
...or tell us you'll get back with more test results in a week (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=38881&view=findpost&p=344130) and never do?
I did post some test files here a long time ago with results and log files
What if you don't exclusively listen to foobar/pc audio and you can hear the difference on your home audio rig more clearly but have no way to prove it because you can not use foobar abx during the time you are hearing those limitations?
There are other methods of doing ABX than foobar, but using foobar with proper time-synchronized and level-matched samples this is very easily controlled. Why can't you connect your PC to your stereo?
I can do that but the quality won't be that great depending on the external dac used which influences being able to tell the difference between lossless and lossy.
- Does the lack of training for searching for specific codec artifacts when doing an abx test mean that one will not be able to hear those differences when just listening to the music for enjoyment from time to time?
Most likely, yes.
No way of proving that though... Thats a variable that could matter depending on how obsessive compulsive a person is.
Its possible that one could be focused on too many specific things trying to find the differences between files when they should listen to other parts of the music instead during an abx test.
Doubtful.
Thats a wrong answer there. I could easily listen to a part without artifacts and not be able to tell the difference then go to a portion of the track with artifacts that are already familiar to me and easily ABX flac and 320 mp3. I could also easily control an abx test by making a person listen through my headphones doing foobar ABX with files that are easy to tell the difference with and ones without artifacts depending on whether I want them to be able to tell the difference between bitrates or not.
Does this really mean that failing to pass a quick ABX test ensures that one cannot hear the difference between these files in the long term?
If you cannot distinguish the difference in the short term, then you probably will not be able to in the long term either, unless you've managed to successfully train yourself, in which case you can test again.
Fair Enough. But then again, you could have just missed the artifacts and then at another time could easily hear it when you were concentrating more
-Is it possible that in certain mental states (different moods, sleepiness, etc...) that certain differences in sound can be heard more easily by people?
...or your ears are stuffed up because of a cold, sure. As I said before, feel free to test again.
Colds are one thing, but there could be other factors that we don't know of yetMy Answers are in BOLD
My Answers are in BOLD
...including enough straw men to build a large bonfire.
Sorry, but seriously: the points you make are relevant to
any listening test. They are in no way unique to ABXing.
The only relevance is that, if you made some of those mistakes in a sighted or non-randomised test, the mistakes would not be caught and would make their way into the published results. Whereas if you make those mistakes under ABX, they're going to be caught (19 times out of 20 on average, for p=0.05). Which means you'll have to go back and find the part/instance/whatever where you
can hear a difference.
Which is far better than reporting "oh yes, I could hear a difference on all the audio content, even when I had a cold or wasn't concentrating" - when in fact there were only some moments where you really heard a difference, and even then you really had to concentrate.
In effect, you're implying that sighted tests are better than ABX because they still let you report audible differences even when/where none exist.
That might be "better" for being able to write 5 pages of prose for Stereophile, but it's not "better" in terms of being able to accurately report reality.
Cheers,
David.
You have a point there but how is it possible to concentrate to double check every second of every track of our entire collections? Why not just save the time and just use lossless instead. Time (to me anyway) is more precious than 1/3 a dime per album. All my OC tendencies and paranoia will come to an end and I can finally just enjoy the music listening to basically all that the recording has to offer... limited to the quality of ones audio system of course I'm sure I'm not alone with this opinion since I see so many people here being very picky on what sounds best. Its simple really... Lossless is still the most transparent to the original CD and everything else(lossy)can vary from junk to great but can never be exactly the same as the original at all times with all possible samples.
It's fine to use lossless.
It doesn't mean mp3 is evil though - or necessarily audibly inferior. That's all.
Cheers,
David.
I notice the same people are less keen to demonstrate the difference signal in the case of 24-bits vs 16-bits, never mind 48kHz sampling vs 96kHz sampling.
I did demonstrate the former at the 2004 AES Convention in the workshop on high-resolution players. Some 24-bit recordings have nothin but random activity in the 7 or [8] LSBs. Others do have real musical information down there.
I'm not sure how you'd reliably tell the difference. I have an interest in how many bits of LPCM include useful information (http://wiki.hydrogenaudio.org/index.php?title=Lossywav) and it's not a trivial question to answer. If the last 7-bits of 24-bit audio were audible, then you increased the gain a little . If they sounded like anything other than white noise, that's quite surprising, though not impossible in contrived signals.
I wasn't discussing audibility. Others on the panel - Vicki Melchior, George Massenburg, Malcolm Hawksford - examined wider and deeper issues raised by the advent of high-resolution media. My brief for the AES workshop was to answer 2 questions: 1) what content is present on 24-bit/96kHz recordings that isn't preserved on CD? and 2) if there is such content, how effective are typical playback systems at preserving that content?
That some (but not all, of course) 24-bit recordings do have signal-related content in the 8 LSBs I thought interesting, given that many people have written that all real-world recordings will have sufficiently high a noise floor to randomize the LSBs below the 16th.
John Atkinson
Editor, Stereophile
I did post some test files here a long time ago with results and log files
I found none relating to the discussion I linked.
I can do that but the quality won't be that great depending on the external dac used which influences being able to tell the difference between lossless and lossy.
Got any objective proof of this? To me is sounds like a desperate excuse (besides being a TOS #8 violation).
No way of proving that though... Thats a variable that could matter depending on how obsessive compulsive a person is.
No, it's not a variable that could matter depending on a person's OCD. You are capable of hearing artifacts or you aren't. If you don't know what to look for you won't find it.
Thats a wrong answer there. I could easily listen to a part without artifacts and not be able to tell the difference then go to a portion of the track with artifacts that are already familiar to me and easily ABX flac and 320 mp3. I could also easily control an abx test by making a person listen through my headphones doing foobar ABX with files that are easy to tell the difference with and ones without artifacts depending on whether I want them to be able to tell the difference between bitrates or not.
Pick the sample and repeat the test. Tell me where it is written that samples from ABX tests must not be chosen by the person being tested. Then tell me where it is written that one failed ABX attempt by one person is universally applicable to everyone.
you could have just missed the artifacts and then at another time could easily hear it when you were concentrating more
Like I said, rinse and repeat. The problem is that many people won't ever reach for the shampoo bottle, ignoring those who proselytize against using shampoo altogether.
Colds are one thing, but there could be other factors that we don't know of yet
It's a wonder science ever progresses.
There were several problems with that study, it wasn't exactly rock solid. One of the most glaring faults being:
Before randomization and during the screening, the placebo pills were truthfully described as inert or inactive pills, like sugar pills, without any medication in it. Additionally, patients were told that “placebo pills, something like sugar pills, have been shown in rigorous clinical testing to produce significant mind-body self-healing processes.”
IOW, they placeboed the placebos.
See here (http://www.sciencebasedmedicine.org/?p=9339) and here (http://blogs.discovermagazine.com/notrocketscience/2010/12/22/evidence-that-placebos-could-work-even-if-you-tell-people-they%E2%80%99re-taking-placebos/) for more details.
Thanks for the links. My point in bringing it in was that there are ethical and truthful ways to manipulate people's expectations and there can be real or perceived benefits in doing so. This study is unable to show real benefits but clearly shows perceived benefits. In high-end audio, it's all about perception. You can change what people perceive by changing their expectations. It's all in their head but that's where perception happens; they really are hearing something different.
And yes it does threaten worldly justice and cosmic balance but there is no requirement for these to be upheld within the confines of an individual's head.
That some (but not all, of course) 24-bit recordings do have signal-related content in the 8 LSBs I thought interesting, given that many people have written that all real-world recordings will have sufficiently high a noise floor to randomize the LSBs below the 16th.
That some 24 bit recordings have signal-related content in the 8 LSBs in no way contradicts the idea that all real-world recordings will have sufficiently high of a noise floor to randomize the quantization error when the 24 bit signal is converted to 16 bits.
Notice that I corrected your misstatement of the idea that "many people" have. ;-)
The idea that some 24 bit recordings have signal-related content in the 8 LSBs that somehow remains detectable when the 24 bit signal is converted to 16 bits with proper randomization of the quantization error is proven by the fact that a properly dithered 16 bit signal may contain audible content that is smaller than the LSB.
In essence John, you've just reiterated the audiophile myth that a 16 bit signal can contain no detectable signal smaller than the LSB. I think you believe what you said. This may be how you *hear* the loss of low level signals when some 24 bit recordings are converted to 16 bits, in your sighted evaluations. ;-)
In high-end audio, it's all about perception. You can change what people perceive by changing their expectations. It's all in their head but that's where perception happens; they really are hearing something different.
A perception that contradicts actuality, such as the false vision of a shimmering lake in the desert, is of course called an illusion.
This is why I say that high end audiophiles who hear changes in their sighted evaluations when there is no actual change are illusional, not delusional.
Their perceptions are similar to those of normal human beings under the same conditions. There is no pathology in their perceptions of sonic differences, given the circumstances.
ABXers can and do perceive non-existent differences when they do sighted evaluations. It's just that we've evolved our means for doing comparisons in the interest of behaving more consistently with actuality.
But playing the difference signal emphasizes that _something_ is removed by lossy codecs, which is a point that I also wanted to make.
John Atkinson
Editor, Stereophile
But since it is a lossy codecs then by _definition_ "something" is removed. Otherwise it wouldn't be a "lossy codecs". I could put on a demonstration that would show that water is wet if I liked, but why would anyone come. What's the point? Since something is by definition removed you can isolate and amplify what is removed and it will obviously be audible on it's own. So what?
The question of interest is not whether something is removed, but whether the something that is removed changes the sound audibly. It certainly changes it physically, but ears have limits. And the evidence seems to be all on the side of the theory that some "lossy codecs" result in differences that are inaudible to human ears. Oddly, people seem to sell these recordings to human beings and not robots or computers. Perhaps that has something to do with who has the money to spend.
That some (but not all, of course) 24-bit recordings do have signal-related content in the 8 LSBs I thought interesting, given that many people have written that all real-world recordings will have sufficiently high a noise floor to randomize the LSBs below the 16th.
I see - I agree that this is interesting for that reason. Do you recall any examples?
I doubt they
"need" more than 16-bits, but they could still be good candidates for a listening test, and in any case may be very good recordings - so always of interest.
Cheers,
David.
That some (but not all, of course) 24-bit recordings do have signal-related content in the 8 LSBs I thought interesting, given that many people have written that all real-world recordings will have sufficiently high a noise floor to randomize the LSBs below the 16th.
I see - I agree that this is interesting for that reason. Do you recall any examples?
Virtually *every* real world recording has "signal-related content" whose amplitude is less than 16 bits, even the ones that were made @ 16 bits, went through production @ 16 bits, and were distributed in a 16 bit format.
In properly dithered digital recordings (which covers pretty much everything modern unless someone screws up) it is always true with recordings of natural sounds that signals whose amplitude is < LSB and even << LSB are present. The caveat "natural sounds" excludes synthesized sounds because it is possible to create a recording with synthesized sounds that lack components whose amplitude is that small. Even that isn't particularly likely, but it could theoretically happen.
The point is that you have to go out of your way to totally loose low level detail. There are two ways that low level detail failes to be perceptible and that is that it gets lost in the noise or masked by louder signals at the same or nearby frequencies. The ear is good at pulling signals out of noise, but it sucks at avoiding masking.
Here's your challenge - how does one make the data that represents low level detail completely go away?
There is an obvious answer which is: use undithered requantization to shorter data words. That's a no-no. Nobody who is serious about quality would intentionally do that, and the world is full of software and equipment that goes out of its way to not do it.
There's another answer - add a noise gate which is a standard production tool, usually a feature of a more generalized dynamics processor. Thing is, that only works during quiet passages - the low level detail in louder signals is still there.
I would sincerely hope that John is pulling your leg. ;-)
Or, he's trying to sell the audiophile myth about only the highest quality, most expensive equipment reproducting low level detail. :-( I think that we've all seen this property ascribed to everything from intereconnects to loudspeakers.
That some (but not all, of course) 24-bit recordings do have signal-related content in the 8 LSBs I thought interesting, given that many people have written that all real-world recordings will have sufficiently high a noise floor to randomize the LSBs below the 16th.
I see - I agree that this is interesting for that reason. Do you recall any examples?
Virtually *every* real world recording has "signal-related content" whose amplitude is less than 16 bits
...I'm sure there's "signal related content" down to the molecular level in the original room, and I'm sure various combinations of microphones and electronics beat 16-bits in some respects.
That's not the point. The claim is that those 8 LSBs of a 24-bit recording contain something that isn't random, and that this "not randomised" nature was demonstrated by listening to them (if I'm following this correctly - I could be wrong!).
I've listened to lots of groups of LSBs
on their own, and you rarely hear anything other than white noise. It depends on how many LSBs, the original bitdepth, and the signal itself of course - but it would still be interesting to know of 24-bit recordings where you hear something other than white noise, because all the ones I've checked (which aren't synthetic, and don't have artificial fade-outs or reverb) just give white noise. I can imagine there may be some which don't quite sound like white noise, and if so, I'd be interested to find them.
Please don't read any more into it than that.
Cheers,
David.
That some (but not all, of course) 24-bit recordings do have signal-related content in the 8 LSBs I thought interesting, given that many people have written that all real-world recordings will have sufficiently high a noise floor to randomize the LSBs below the 16th.
I see - I agree that this is interesting for that reason. Do you recall any examples?
Virtually *every* real world recording has "signal-related content" whose amplitude is less than 16 bits
...I'm sure there's "signal related content" down to the molecular level in the original room, and I'm sure various combinations of microphones and electronics beat 16-bits in some respects.
That's not the point. The claim is that those 8 LSBs of a 24-bit recording contain something that isn't random, and that this "not randomised" nature was demonstrated by listening to them (if I'm following this correctly - I could be wrong!).
I've listened to lots of groups of LSBs on their own, and you rarely hear anything other than white noise. It depends on how many LSBs, the original bitdepth, and the signal itself of course - but it would still be interesting to know of 24-bit recordings where you hear something other than white noise, because all the ones I've checked (which aren't synthetic, and don't have artificial fade-outs or reverb) just give white noise. I can imagine there may be some which don't quite sound like white noise, and if so, I'd be interested to find them.
Please don't read any more into it than that.
Now that you've explained what you are doing, I think I can explain your results.
As I read it, you've been stripping off the low order bits of recordings, at points where the amplitude of the recording is greater than just the bits you are stripping off. For example there may be a portion of a 24 bit integer audio track whole amplitude at some point is say the 16 low order bits. You've been stripping off lets say the 6 lowest order bits. The results are white noise. I find that to be completely predictable because of what you're doing, not necessarily due to the content of the audio track. What you are hearing is rather massive quantization noise because of the odd kind of requantizing that you are doing, and also because whatever kind of re-quantizing you are doing, you're not randomizing the quantization error.
One of the things I've learned is that low level detail is not restricted to the low order bits.
For example, lets take a test tone composed of a high amplitude signal (-6 dB FS) at 1 KHz, and a very low amplitude signal
at 500 Hz (-60 dB FS).
The question is which bits contain the low level signal. The answer is that all of the bits that represent the -6 dB FS signal are altered by the presence of the -60 dB signal. If you want to hear the -60 dB signal, you need to listen to far more than just however many few low order bits. You need to listen to *all* of the bits that are affected by the -6 dB FS signal in order to hear the -60 dB FS signal.
Now If I've misjudged your situation, my apologies, but I'm just trying to read what you write and understand it, and share my experience with this sort of thing.
Sounds like David's essentially doing an Atkinson style difference test between 24-bit and truncated 16-bit version of the same recording. He wants to see if anything is lost by this data reduction. I think it is interesting and non-intuitive that he hasn't found anything. Especially so considering this difference process does not dither properly.
In high-end audio, it's all about perception. You can change what people perceive by changing their expectations. It's all in their head but that's where perception happens; they really are hearing something different.
A perception that contradicts actuality, such as the false vision of a shimmering lake in the desert, is of course called an illusion.
This is why I say that high end audiophiles who hear changes in their sighted evaluations when there is no actual change are illusional, not delusional.
Their perceptions are similar to those of normal human beings under the same conditions. There is no pathology in their perceptions of sonic differences, given the circumstances.
ABXers can and do perceive non-existent differences when they do sighted evaluations. It's just that we've evolved our means for doing comparisons in the interest of behaving more consistently with actuality.
Do audiophiles accept actuality as a useful concept?
Sounds like David's essentially doing an Atkinson style difference test between 24-bit and truncated 16-bit version of the same recording. He wants to see if anything is lost by this data reduction. I think it is interesting and non-intuitive that he hasn't found anything. Especially so considering this difference process does not dither properly.
Yes, exactly. I'd imagine there may be recordings where you find something, which is why I'm interested to know which ones they are, if they exist.
On a separate topic, the difference signal due to lossyWAV's basic undithered no-noise-shaping truncation is always something like blocks of white noise. In the case of a couple of pure tones, it wouldn't remove any bits at all - it sees a -xxdB noise floor beyond the tones and leaves well alone.
lossyWAV sometimes wants to keep more than 16-bits though.
Cheers,
David.
I'm sure there's "signal related content" down to the molecular level in the original room, and I'm sure various combinations of microphones and electronics beat 16-bits in some respects.
That's not the point. The claim is that those 8 LSBs of a 24-bit recording contain something that isn't random, and that this "not randomised" nature was demonstrated by listening to them (if I'm following this correctly - I could be wrong!).
As I said in an earlier posting, I didn't comment in my AES presentation on the audibility of what was happening in bits 17-24, only that while there was often random bit activity, with some recordings there was some signal-correlated activity in the 8 LSBs.
John Atkinson
Editor, Stereophile
But playing the difference signal emphasizes that _something_ is removed by lossy codecs, which is a point that I also wanted to make.
But since it is a lossy codecs then by _definition_ "something" is removed. Otherwise it wouldn't be a "lossy codecs".
Of course, and I expect that point is obvious to all on HA. But to the public at large, who have been told that even low-bit-rate satellite radio is "CD quality," the difference between lossless and lossy bit-rate reduction is not as clear as you might expect. To judge by some of the emails I receive, that difference isn't even as clear as I would expect among Stereophile's readership. That was the point I was making.
John Atkinson
Editor, Stereophile
In high-end audio, it's all about perception. You can change what people perceive by changing their expectations. It's all in their head but that's where perception happens; they really are hearing something different.
A perception that contradicts actuality, such as the false vision of a shimmering lake in the desert, is of course called an illusion.
This is why I say that high end audiophiles who hear changes in their sighted evaluations when there is no actual change are illusional, not delusional.
Their perceptions are similar to those of normal human beings under the same conditions. There is no pathology in their perceptions of sonic differences, given the circumstances.
ABXers can and do perceive non-existent differences when they do sighted evaluations. It's just that we've evolved our means for doing comparisons in the interest of behaving more consistently with actuality.
Do audiophiles accept actuality as a useful concept?
The tragedy is that this is an interesting question.
Sounds like David's essentially doing an Atkinson style difference test between 24-bit and truncated 16-bit version of the same recording. He wants to see if anything is lost by this data reduction. I think it is interesting and non-intuitive that he hasn't found anything. Especially so considering this difference process does not dither properly.
Yes, exactly. I'd imagine there may be recordings where you find something, which is why I'm interested to know which ones they are, if they exist.
One non-intutive thing about what you are actually testing, is that the 16 bit version can reproduce signals whose peak amplitude is << 1 (16 bit) LSB.
So, when you take the difference of these two signals, you're cancelling out a great deal of musical information whose amplifude is << 1 (16 bit) LSB. That is one explanation why all you get is noise.
It's fine to use lossless.
It doesn't mean mp3 is evil though - or necessarily audibly inferior.
Mr. Atkinson:
this is all you ever have to say in your 'dems', to tell the complete truth most efficiently.
Of course, and I expect that point is obvious to all on HA. But to the public at large, who have been told that even low-bit-rate satellite radio is "CD quality," the difference between lossless and lossy bit-rate reduction is not as clear as you might expect. To judge by some of the emails I receive, that difference isn't even as clear as I would expect among Stereophile's readership. That was the point I was making.
John Atkinson
Editor, Stereophile
But to then NOT tell the public at large (or at least, the sample in the room with you, or in your letters column ) that detection of audible difference between lossy and lossless is not necessarily predictable from the difference signal, would be disingenuous at best. So I'm sure you make a point of mentioning that. Right? Because you're interesting in educating the public, right?
One non-intutive thing about what you are actually testing, is that the 16 bit version can reproduce signals whose peak amplitude is << 1 (16 bit) LSB.
So, when you take the difference of these two signals, you're cancelling out a great deal of musical information whose amplifude is << 1 (16 bit) LSB. That is one explanation why all you get is noise.
I think we're going around in circles here. Of course if it's correctly dithered the difference is noise - that's the whole point of dither.
Forget dither. Forget 16-bits. Forget what's audible. Forget everything!
1) Take a 24-bit recording. Play only the 8 LSBs, shifted into the 8 MSBs so you can hear them. What do they sound like?
2) In comparison, take a 16-bit recording. Play only the 8 LSBs, shifted into the 8 MSB to match the above. What do they sound like?
IME 1) = always white noise, 2) = sometimes white noise, but sometimes distorted mess with
something vaguely related to the original signal clearly audible.
Cheers,
David.
As I said in an earlier posting, I didn't comment in my AES presentation on the audibility of what was happening in bits 17-24, only that while there was often random bit activity, with some recordings there was some signal-correlated activity in the 8 LSBs.
What do you mean? I'm not trying to argue or nitpick - but honestly, what do you mean by this?
I know what "signal-correlated" means, but it's not immediately obvious to me how you make this judgement in this case.
I can easily give examples where there
is (specific synthetic signals), and where there isn't (a 16-bit recording with 8 bits of noise added) - but I'm interested to know how you can even begin to make this call if I give you a 24-bit recording from an unknown source.
If we were talking about dither, and whether a specific type (or lack of) dither left signal-correlated distortion due to word length reduction, then there would be at least three things we could fall back on:
1) Mathematical proof of what the dither+quantisation does to the signal.
2) Processing of test signals and using a) waveform and/or b) spectral analysis on the results.
3) Listening - to a) the result and/or b) the difference signal
OK - I cheated - that's actually five things. Only (1) is actually conclusive, but (2a+b) and (3a+b) can still be good indicators or at least demonstrators.
I can't see how you can perform (1) for a given 24-bit recording. We can try 2 and 3 though.
Cheers,
David.
One non-intutive thing about what you are actually testing, is that the 16 bit version can reproduce signals whose peak amplitude is << 1 (16 bit) LSB.
So, when you take the difference of these two signals, you're canceling out a great deal of musical information whose amplified is << 1 (16 bit) LSB. That is one explanation why all you get is noise.
One good one explanation is all that is needed!
I think we're going around in circles here.
Not me. I've got one good explanation that is relevant and correct.
Of course if it's correctly dithered the difference is noise - that's the whole point of dither.
So why tie yourself in knots and leave reason and logic behind?
Forget dither.
Seems illogical to me.
Forget 16-bits. Forget what's audible. Forget everything!
Now, we would appear to be in Atkinson territory - Fantasy Land! ;-)
1) Take a 24-bit recording. Play only the 8 LSBs, shifted into the 8 MSBs so you can hear them. What do they sound like?
That depends on whether there was data in the 16 MSBs that you threw away. If there was data there, than you hear the result of clipping. You have a mess.
2) In comparison, take a 16-bit recording. Play only the 8 LSBs, shifted into the 8 MSB to match the above. What do they sound like?
That depends on whether there was data in the 8 MSBs that you threw away. If there was data in there, than you hear the result of clipping. You have a mess.
If there the signals were typical and there was data that you threw away in either case, then the following is *not* true.
IME 1) = always white noise, 2) = sometimes white noise, but sometimes distorted mess with something vaguely related to the original signal clearly audible.
Looks to me like a gedanken experiment that went kerflooie because it did not treat the most common situation, which is data in the MSBs.
Besides all the swagger I think Arnold's point, that you get massive clipping of those components belonging to the stripped MSBs, really is a show-stopper issue. A lot of what you hear in the isolated 8 LSBs is just that.
Would transforming to the frequency domain, stripping all components above a certain threshold, and converting back give more meaningful results?
As I said in an earlier posting, I didn't comment in my AES presentation on the audibility of what was happening in bits 17-24, only that while there was often random bit activity, with some recordings there was some signal-correlated activity in the 8 LSBs.
What do you mean? I'm not trying to argue or nitpick - but honestly, what do you mean by this?
I know what "signal-correlated" means, but it's not immediately obvious to me how you make this judgement in this case.
Exactly. The only reliable way I know of to establish a system's perforamance with low-level signals is to resort to contrived signals, including test tones.
Now that fade outs and reverb tails have been excluded by some, its really hard to find clearly identifiable low level signals that everybody is going to like.
Guys guys, this is getting way out of hand. I pursue one little aside of an aside in a thread - for reasons that have nothing to do with the point of the thread, and (as it happens) nothing to do with any potential audible advantages of 24-bit vs 16-bit audio, and Arny repeatedly tries to give me a lecture on dither.
Stop trying to answer questions I haven't asked.
I am looking for real acoustic recordings that are so clean that there's no hope in hell of them self dithering at 16-bits. JA's aside implied that he'd found some (if you think it through). Or else he has found some other way of determining what's above and below the noise floor - which in itself would be worth investigation (because I think that's a very complicated topic).
Either such recordings exist, or else dither is almost pointless (think about it). I don't have many 24-bit recordings, but of the ones I have to hand on my PC, none appear anywhere near that "clean". None appear to need those 24-bits (even ignoring the limitations of the human ear); none appear to even need dither. So I was wondering which recordings JA had found which has something non-random-ish in the 8 LSBs, and how he determined this fact.
That's it. No more. No less.
I'm beginning to wish I'd just PM'd JA!
Cheers,
David.
I am looking for real acoustic recordings that are so clean that there's no hope in hell of them self dithering at 16-bits.
AFAIK no such thing exists. The current record holder among commercial recordings a tad less than 15 bits.
I know how to make such a recording, but the method would be really not meet the stated criteria.
Either such recordings exist, or else dither is almost pointless (think about it).
Even though such recordings don't seem to exist, there is still plenty of use for dither. Many steps in audio production still need it.
OTOH, let me explain how to make a recording of a real world performance that for sure needs dither. I just make the widest dynamic range recording I know of, but make it with say 10 dB headroom which is merely good practice.
The peak level moves from FS to FS -10 dB, and the quietest passage moves from FS -88 dB to FS - 98 dB. The need for dither is clear. The need to produce the recording with > 16 bit resolution at the live recording session becomes clear.
Expand on that a little and you will see that it is easy to make recordings with music-related information in the 8 LSBs of 24 bits. Just convert a 16 bit recording to 24 bits and attenuate by 8 bits which is about 48 dB.
Other ways to do this exist, but they net out to about the same thing.
If you want to be stuffy about having music in the upper 8 bits, learn how to do dyanmic range expansion with 24 bit precision. ;-)
But to the public at large, who have been told that even low-bit-rate satellite radio is "CD quality," the difference between lossless and lossy bit-rate reduction is not as clear as you might expect. To judge by some of the emails I receive, that difference isn't even as clear as I would expect among Stereophile's readership. That was the point I was making.
But to then NOT tell the public at large (or at least, the sample in the room with you, or in your letters column ) that detection of audible difference between lossy and lossless is not necessarily predictable from the difference signal, would be disingenuous at best. So I'm sure you make a point of mentioning that.
Yes I did. (And didn't you already ask me this question?)
John Atkinson
Editor, Stereophile
As I said in an earlier posting, I didn't comment in my AES presentation on the audibility of what was happening in bits 17-24, only that while there was often random bit activity, with some recordings there was some signal-correlated activity in the 8 LSBs.
What do you mean? I'm not trying to argue or nitpick - but honestly, what do you mean by this?
I know what "signal-correlated" means, but it's not immediately obvious to me how you make this judgement in this case.
Set the 16 MSBs to 0 and examine what bits 17 to 24 contain in isolation.
Regarding dither, I note you said in another message that you were "looking for real acoustic recordings that are so clean that there's no hope in hell of them self dithering at 16-bits." I looked into this at length several years ago. The problem is that with low-noise mikes and preamps, recordings of acoustic music tend to have have brown or red noise floors, so while the RMS audioband level of the noise is above the 16-bit quantization floor, you can't assume that there will be spectral content at the LSB level to provide optimal dithering. If not, then of course, there is no dithering action. If the noise has the right spectral content but is somewhat too high in level, with an undithered quantizer you get noise modulation of the signal, something that Stanley Lipshitz demonstrated at an AES convention many years ago.
So why not always use dither of the right PDF and level, just in case the signal might not be self-dithering? It can't do any harm and it might be needed.
Having said that, when Keith Howard wrote an article on the subject for Stereophile - http://www.stereophile.com/features/705dither/index.html (http://www.stereophile.com/features/705dither/index.html) - he disagreed somewhat: "The message I hope you take away from it is that dither—for all its wondrous ability to confer analog-like behavior on digitized signals—should be applied with care, particularly at the 16-bit level. In converting some 24-bit files to 16-bit it may be unnecessary to use dither, and the sound quality may benefit from its deletion."
Note, BTW, that Keith also found many 24-bit commercial recordings not to have signal-related activity below the 16th bit, which is one of the points I made in my AES presentation.
Thanks for the feedback, David.
John Atkinson
Editor, Stereophile
Set the 16 MSBs to 0 and examine what bits 17 to 24 contain in isolation.
The irony of trying to find very small signals in the face of such massive clipping!
The fallacy of believing that small signals are only contained in the LSBs.
Not exactly reasonble ways to find the Holy Grail.
Set the 16 MSBs to 0 and examine what bits 17 to 24 contain in isolation.
The irony of trying to find very small signals in the face of such massive clipping!
What "clipping"?
The fallacy of believing that small signals are only contained in the LSBs.
What fallacy? It was purely the nature of the contents in bits 17-24 that I was asked to examine.
John Atkinson
Editor, Stereophile
Cutting LSBs is fine, but cutting MSBs leads to serious distortion due to clipping. It's not trivial to understand why this happens but you'll get to it when you think about Arnold's example of two concurrent waves with different volume and how the louder one also modifies the LSBs. When you cut out the MSBs in that case, you won't be left with only the quieter wave but the latter + clipped artifacts of the louder wave.
I read the discussion about 24bit with very high interest cause it suddenly pops up everywhere on the net. Most likely cause of Apples decision.
It would be great to have a thread with some of the infos that were already posted in here could be started. If anyone has an idea?
On Slimdevices there are 2 threads for example discussing this subject.
But now that someone answered with a further link to "computeraudiophile" i thought we need a thread here i can link that people to
Cutting LSBs is fine, but cutting MSBs leads to serious distortion due to clipping. It's not trivial to understand why this happens but you'll get to it when you think about Arnold's example of two concurrent waves with different volume and how the louder one also modifies the LSBs. When you cut out the MSBs in that case, you won't be left with only the quieter wave but the latter + clipped artifacts of the louder wave.
A few points:
- Clipped white noise is still white noise.
- If we hear anything that is signal dependent in the LS bits, that's very strong evidence that there's information in those bits - reduced entropy indicates presence of information.
- If we only hear noise it does not necessarily eliminate the possibility that there's information there - there are many cases where useful information is white.
Has anyone attempted this with the samples provided which are actually on-topic to this discussion?
I remember JA offering recordings with non-random content below 16 bit, but I do not remember seeing any actual links. Have I missed anything?
http://www.hydrogenaudio.org/forums/index....st&p=743399 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86649&view=findpost&p=743399)
Cutting LSBs is fine, but cutting MSBs leads to serious distortion due to clipping.
Not in my experience. Have you tried this for yourself with a recording that has correlated information in bits 17-24? I think the word "clipping" is being misused here.
As "Notat" wrote in a recent message: "If we hear anything that is signal dependent in the LS bits, that's very strong evidence that there's information in those bits - reduced entropy indicates presence of information."
That's the point I was making at the AES workshop: that with _some_ 24-bit recordings, there are valid signal data in bits 17-24.
John Atkinson
Editor, Stereophile
I think the word "clipping" is being misused here.
The phenomenon is not only something comparable to clipping but
exactly the correct term. It might be easier to comprehend for you when you realize that stripping the MSBs is equivalent to digital amplification (multiplication by 2^n, n=number of bits), hard into 0dB, followed by attenuation (division by 2^n) back to the original level. This is a paramount example for digital clipping, there is not the slightest misuse of the word.
Cutting LSBs is fine, but cutting MSBs leads to serious distortion due to clipping.
Not in my experience.
You need to open your eyes up and look at what happens to *any* waveform when you start stripping off the MSBs.
Have you tried this for yourself with a recording that has correlated information in bits 17-24?
Yup.
I think the word "clipping" is being misused here.
I think that if you actually look at a wave that has had the first few MSBs zeroed out, I'll bet that you'll say to yourself: "Why that wave has been clipped"!
I think the word "clipping" is being misused here.
The phenomenon is not only something comparable to clipping but exactly the correct term. It might be easier to comprehend for you when you realize that stripping the MSBs is equivalent to digital amplification (multiplication by 2^n, n=number of bits), hard into 0dB, followed by attenuation (division by 2^n) back to the original level. This is a paramount example for digital clipping, there is not the slightest misuse of the word.
Thank you. Yes, it does appear I was incorrect. The question then becomes: is the residue random in nature, ie white noise, which when clipped is unchanged, or is it still signal-related, in which case the 8 LSBs did contain audio data and not random bit switching?
John Atkinson
Editor, Stereophile
The phenomenon is not only something comparable to clipping but exactly the correct term. It might be easier to comprehend for you when you realize that stripping the MSBs is equivalent to digital amplification (multiplication by 2^n, n=number of bits), hard into 0dB, followed by attenuation (division by 2^n) back to the original level. This is a paramount example for digital clipping, there is not the slightest misuse of the word.
Not exactly correct. Clipping should imply saturation for samples outside the allowed range. What happens here is that out-of-range samples are wrapped back into range. Sort of an amplitude aliasing thing. Sounds very nasty but does not totally whiten or clobber the information.
Notat is correct. It's not clipping at all.
If you clipped it like this, you'd certainly hear something related to the music afterwards, whatever the original contents of the 8LSBs.
Cheers,
David.
Could someone enlighten me?
1. If I amplify a sequence of samples n bits over what their containers can hold, I get digital clipping. The top n bits fall over board. I guess no one wants to challenge that.
2. If I attenuate the same sequence afterwards by the same amount, the resulting sequence will be identical to the original sequence with (n) MSBs blanked and (total - n) LSBs untouched.
How does 2. invalidate the term "clipping"?
If the DAC has a little headroom above its rated bitdepth (many do) 1. and 2. should lead to comparable spectral components after DA conversion.
The phenomenon is not only something comparable to clipping but exactly the correct term. It might be easier to comprehend for you when you realize that stripping the MSBs is equivalent to digital amplification (multiplication by 2^n, n=number of bits), hard into 0dB, followed by attenuation (division by 2^n) back to the original level. This is a paramount example for digital clipping, there is not the slightest misuse of the word.
Not exactly correct. Clipping should imply saturation for samples outside the allowed range. What happens here is that out-of-range samples are wrapped back into range. Sort of an amplitude aliasing thing. Sounds very nasty but does not totally whiten or clobber the information.
I see your point, and its important as far as it goes. However, we're still talking about massive ruination of the wave form and applying huge amounts of nonlinear distoriton. I know of no standard that accepts or recommends applying massive nonlinear distortion to a signal in order to recover information about small components of it.
BTW there are natural situations that have similar results such as old-style ladder DACs that are missing a lot of codes that are next to each other. I don't recall anybody ever recomending using broken DACs to measure low level detail like crossover distortion. The usual recommendation was to fix them so that they were linear and monotonic!
Removing MSB bits will result in a very distorted sound wave with large transients (square waves), a simple representation is a 8 bit signal (0-255 for simplicity with no -), if the upper 8th bit is removed then anything above 128 will jump down to signal-128, so 126 and 127 would be ok, 128 would become 0 and 129 would become 1.
Notat is correct. It's not clipping at all.
It is equivalent to listening to the isolated quantisation distortion introduced by truncating the 24b source
material to 16b.
If the residue is / sounds like just noise this means that the 24b original had a sub-16b innate noise floor, overwhelming any payload signal in the lower 8 bits. And that seems to be exactly what you get with the sample referred to above, admittedly tried only in a noisy office environment and with cheap open headphones.
(edit)
Doing the same with 24b versus 8b reveals a crackling residue. Re-doing 24b versus 16b now (lunchtime, hence quieter environment and louder replay levels) reveals a high-pitched tinnitus-like signal that is keyed-on and -off, mostly in the right channel. So the residue is not strictly white.
I was wondering, how am I able to set the top msb to 0?
Is there a tool that can do that?
I mean without amplifing the signal over 0dB and then reducing gain.
Truncate to 16 bit. Expand again to 24 bit (i.e. all 8 LSBs are now zero). Subtract from 24 bit source (all 16 MSBs are now zero).
Truncate to 16 bit. Expand again to 24 bit (i.e. all 8 LSBs are now zero). Subtract from 24 bit source (all 16 MSBs are now zero).
Thanks Werner.
I tried it with some drum loops I got from the net and I hear besides hudge amounts of noise some drum sounds to.
Has anyone attempted this with the samples provided which are actually on-topic to this discussion?
http://www.hydrogenaudio.org/forums/index....st&p=743399 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86649&view=findpost&p=743399)
Yes.
Bits 17-24 sound like white noise to me.
I've uploaded them next to the original...
http://www.hydrogenaudio.org/forums/index....showtopic=86738 (http://www.hydrogenaudio.org/forums/index.php?showtopic=86738)
(It's still an 88.2kHz 24-bit file, though there are a lot of zeros in there, hence the comparatively small FLAC filesize for a noise-like signal which hits digital full scale - wasted_bits is very useful!).
Cheers,
David.
EDIT: I posted this without seeing the last 7 posts in this thread.
It is equivalent to listening to the isolated quantisation distortion introduced by truncating the 24b source material to 16b.
I agree.
Doing the same with 24b versus 8b reveals a crackling residue. Re-doing 24b versus 16b now (lunchtime, hence quieter environment and louder replay levels) reveals a high-pitched tinnitus-like signal that is keyed-on and -off, mostly in the right channel. So the residue is not strictly white.
I don't think your software has done what you think it has. See the sample I posted.
FWIW In Cool Edit Pro, if you convert 24>16, and then subtract the 16 from the 24, you don't just get the 8LSBs. The undithered 24>16 in Cool Edit Pro isn't quite the simple bit discarding that you might expect, but actually rounds all values up (except zero!).
Even so, you still get something which
sounds like it should (except for the jumps in amplitude on any zeros - which is still inaudible in a noisy signal), even though mathematically it's wrong.
Cheers,
David.
I tried it with some drum loops I got from the net and I hear besides hudge amounts of noise some drum sounds to.
I don't think there's any doubt that you'll hear something for artificial signals, generated at 24-bits.
You can generate an artificial signal with real signal-correlated data to as many bits as you want.
A sine wave generated accurately to 32-bits or 64-bits would still have something signal-correlated in the last 8 bits, showing that 24 or even 56 bits just aren't enough.
Cheers,
David.
Bugger. My tool's manual claims it truncates, but the resultant difference file sounds a bit dirtier than yours, which indeed is a rather clean kind of noise. To Be Characterised.
I tried it with some drum loops I got from the net and I hear besides hudge amounts of noise some drum sounds to.
I don't think there's any doubt that you'll hear something for artificial signals, generated at 24-bits.
You can generate an artificial signal with real signal-correlated data to as many bits as you want.
A sine wave generated accurately to 32-bits or 64-bits would still have something signal-correlated in the last 8 bits, showing that 24 or even 56 bits just aren't enough.
Cheers,
David.
It was a recording of the famous Amen drum loop, so its not generated but recorded from LP.
It was a recording of the famous Amen drum loop, so its not generated but recorded from LP.
I never knew that drum track was called that! The wikipedia page (http://en.wikipedia.org/wiki/Amen_break) is a goldmine!
I'm surprised there's anything important in the 12th bit, never mind the 20th. I suspect something strange is happening, but who knows.
Cheers,
David.
It is equivalent to listening to the isolated quantisation distortion introduced by truncating the 24b source
material to 16b.
If the residue is / sounds like just noise this means that the 24b original had a sub-16b innate noise floor, overwhelming any payload signal in the lower 8 bits.
I think it is safe conclude there's information in the LSBs if we hear a correlated signal there. I'm not convinced the converse is true. Just because we don't hear anything doesn't mean there's nothing important there. This is a non-linear process and so difficult to model how the signal is transformed. There are processes which will make information sound like white noise. Encryption is the obvious example.
Also be careful when discussing noise floor. There almost always is useful information below the noise floor.
There almost always is useful information below the noise floor.
Below the summed/integrated noise: yes.
Below the spectrally-local noise density floor: no, not really. Try listening to a fade to
noise while monitoring it on a decent real time spectrometer.
There almost always is useful information below the noise floor.
Below the summed/integrated noise: yes.
Below the spectrally-local noise density floor: no, not really. Try listening to a fade to
noise while monitoring it on a decent real time spectrometer.
Agreed.
And what you
see depends largely on the signal content
and the settings of the FFT (or similar).
You can use a longer FFT to "see" a pure tone further into the noise.
The ear's auditory filters cannot be adjusted in the same way .
In other words, it's quite possible to see something which is entirely inaudible - the apparently "lower" noise floor actually masks the "higher level" tone within the auditory filter in the ear.
Cheers,
David.
I think it is safe conclude there's information in the LSBs if we hear a correlated signal there. I'm not convinced the converse is true.
I agree. (I've got to stop doing that!).
This is actually quite complicated. Hence my interest in what on the surface seemed quite a simple statement from JA.
It's this complicated...
http://www.hydrogenaudio.org/forums/index....st&p=746571 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86649&view=findpost&p=746571)
Cheers,
David.
And what you see depends largely on the signal content and the settings of the FFT (or similar).
You can use a longer FFT to "see" a pure tone further into the noise.
The ear's auditory filters cannot be adjusted in the same way .
In other words, it's quite possible to see something which is entirely inaudible - the apparently "lower" noise floor actually masks the "higher level" tone within the auditory filter in the ear.
Cheers,
David.
The ear's maximum integration time is circa 200 milliseconds, or perhaps less, but more importantly, the bandwidth of the ear's filter is what really matters here, and they are quite broad, compared to an FFT, so in fact it is easy to develop a signal which stands out like a lightbulb in an FFT which is totally inaudable under any circumstances above the masking noise.
In an FFT, you're plotting not energy per root Hz, but rather energy per BIN. All else follows, when you realize that noise gain is RMS, while signal gain is amplitude sum.
I guess no one is going to present any real world 24-bit recordings with non-noise-like 8 LSBs?
I guess no one is going to take a stab at defining what is/isn't "signal related information" in those last 8 LSBs either?
You're all cowards
Cheers,
David.
I guess no one is going to present any real world 24-bit recordings with non-noise-like 8 LSBs?
I guess no one is going to take a stab at defining what is/isn't "signal related information" in those last 8 LSBs either?
You're all cowards
Nahh, I showed that the presnce/non presence of signal related information in the LSBs is not the important thing.
The capability to transmit high resolution information of a digital signal is not just in the LSBs. The capability to transmit high resolution information is in the uniform spacing between *all* of the different signal levels that it represents. The MSBs are just as important. If you are building a DAC the LSBs have to be more accurate than the LSBs by far.
The idea that the LSBs are the unique home of low level signals is yet another audiophile myth.
The noise floor of sound in the natural world is set by the Brownian Motion of air.
You're still over thinking it.
JA claimed signal-correlated activity in those last 8 LSBs. (http://www.hydrogenaudio.org/forums/index.php?showtopic=86649&st=300&p=746446&#entry746446)
I'd like to see a recording that shows this. And maybe consider how one decides what's signal correlated or not (http://www.hydrogenaudio.org/forums/index.php?showtopic=86649&st=300&p=746571&#entry746571).
I really don't want another lecture on why I shouldn't be interested in this.
EDIT: Sorry to sound harsh, but you keep jumping in and answering a different question, and talking to me like a grandparent would talk to their three year old. It's getting really tiring.
Cheers,
David.
Correlated to what?
1. Correlated to the analog input fed into the ADC? That's easy, every bit is correlated within the full extend an ADC's capabilities, i.e. the complete word length minus the ADC's noise, which is measurable. You'll find at least 120 dB worth of undoubtedly correlated content within the digital output of a good ADC.
2. Correlated to the acoustic phenomenon before any (analog) electronic capture? That's not as easy, but totally really unrelated to the MSB/LSB juggling, that has been going on here lately. The question remains the same, whether the result is digitally captured or not.
Correlated to what?
1. Correlated to the analog input fed into the ADC? That's easy, every bit is correlated within the full extend an ADC's capabilities, i.e. the complete word length minus the ADC's noise, which is measurable. You'll find at least 120 dB worth of undoubtedly correlated content within the digital output of a good ADC.
Firstly, let me say that I think you're hitting the point now. There was a statement that something was signal-correlated, without (AFAICT) any real details about what that means, or how we could prove/disprove it. So, yes, this is exactly my point. The statement is ill defined and difficult to prove either way.
But running with your question, I think what you've said above is more or less true. But that begs another question: are the bits
below the ADC's noise floor "correlated" to what was fed into it?
What about the noise due to other electronics in the chain? Are the bits below
this noise floor correlated to the signal that came out of the microphone (before other noise was added)?
What about the noise in the room at the point where the microphones were placed (or at least any white-noise-like component of it)? Are the bits below
this noise floor correlated to the sound that was in the room a bit closer to the instruments themselves (i.e. where the SNR was a little higher?).
I think, in the terms WernerO put it...
There almost always is useful information below the noise floor.
Below the summed/integrated noise: yes.
Below the spectrally-local noise density floor: no, not really. Try listening to a fade to
noise while monitoring it on a decent real time spectrometer.
...the correct answer is no. While it's true that you can twiddle the spectrum analysis to pull out this or that, there's an uncertainty-principle time/frequency relationship beyond which you cannot pull something out of noise. Beyond this, the thing isn't there. Or if it is, there's no way of determining whether it really is or isn't, so basically it isn't!
So I propose there is a level below the noise at which things below the noise really are lost, if you accept some finite limit on integration time / FFT length.
This means that, for a given spectral noise level, there are only so many bits you need to store all the real data that's in the signal. If you use any more bits, those extra bits aren't storing anything that is in any way useful.
I could be wrong of course, but that's the way I see it. And from this it follows on that a given recording only ever needs X bits. You don't even need to consider replay level, human ears, masking, or even dither. The only caveat is that you'll raise the noise floor by a predictable amount (quantisation noise). If you don't want to do that, don't quantise - and for goodness sake don't process the signal or put it in the analogue domain .
So, back to the original claim. Are there 24-bit recordings where those last 8-bits contain something above the spectral noise floor, hence signal-correlated, hence at least potentially useful? I'm sure there must be, but I'm still waiting for a pointer to one.
Cheers,
David.
You're still over thinking it.
Nope.
I see a lot of people trying to answer an audiophile myth. Dispel the myth if you want the answer you seek.
JA claimed signal-correlated activity in those last 8 LSBs. (http://www.hydrogenaudio.org/forums/index.php?showtopic=86649&st=300&p=746446&#entry746446)
I'd like to see a recording that shows this.
First state the problem in a reasonable way. This low order bit thing is a red herring! It is one of JA's audiophile myths,
If low order bits were so important to reproduciing small details, DAC monotonicity from rail to rail would not have been such a concern back in the days when it wasn't inherent.
Here's a good rule: Bits are not nearly as importeant as integrity of the data that they represent. It should be a short step from there to a complete dropping of discussion of just low order bits.
And maybe consider how one decides what's signal correlated or not (http://www.hydrogenaudio.org/forums/index.php?showtopic=86649&st=300&p=746571&#entry746571).
I really don't want another lecture on why I shouldn't be interested in this.
You seem to be edging towards how things are, here:
1. Correlated to the analog input fed into the ADC? That's easy, every bit is correlated within the full extend an ADC's capabilities, i.e. the complete word length minus the ADC's noise, which is measurable. You'll find at least 120 dB worth of undoubtedly correlated content within the digital output of a good ADC.
EDIT: Sorry to sound harsh, but you keep jumping in and answering a different question, and talking to me like a grandparent would talk to their three year old. It's getting really tiring.
In my view you keep throwing the cereal bowl onto the floor. ;-)
You're still over thinking it.
Nope.
I see a lot of people trying to answer an audiophile myth. Dispel the myth if you want the answer you seek.
JA claimed signal-correlated activity in those last 8 LSBs. (http://www.hydrogenaudio.org/forums/index.php?showtopic=86649&st=300&p=746446&#entry746446)
I'd like to see a recording that shows this.
First state the problem in a reasonable way. This low order bit thing is a red herring! It is one of JA's audiophile myths
Whether it's a red herring or not depends on what you are trying to find out.
Signal correlated content in the LSBs is certainly not a "myth" - for a given recording, it can only be:
a) demonstrably true, or
b) not demonstrably true.
A claim that it has some relevance to what we hear, or an implication that it proves 16-bits aren't enough, or a suggestion that this signal is the difference between a 24-bit master and a correctly dithered 16-bit version -
those would be audiophile myths. Those are unsubstantiated claims and/or just plain wrong.
But I am not going there, neither am I intending to go there.
So please, for the last time, and I'm asking nicely here - please stop replying to this question with answers to different questions that I haven't asked. Thank you.
Cheers,
David.
P.S. If you have time to waste, please go and download lossyWAV and listen to what it takes away. Disable noise shaping, or grab an older version without it.
I see a lot of people trying to answer an audiophile myth. Dispel the myth if you want the answer you seek.
JA claimed signal-correlated activity in those last 8 LSBs. (http://www.hydrogenaudio.org/forums/index.php?showtopic=86649&st=300&p=746446&#entry746446)
I'd like to see a recording that shows this.
All of them that are well-made digital recordings.
That there is signal-related activity in the low order bits is a truism.
Consider a recording of 20 Hz FS. The low order bits are flailing away to ensure that the signal has low noise and distortion. Of course their flailing is signal-related. Reduce the signal to zero and their flailing will just be due to dither. Increase the signal and their flailing will be as required to preserve the low noise and distortion of the 20 Hz sine wave. Since there will now be quantization distortion that needs to be decorrelated, the flailing will change, subtly.
...the correct answer is no. While it's true that you can twiddle the spectrum analysis to pull out this or that, there's an uncertainty-principle time/frequency relationship beyond which you cannot pull something out of noise. Beyond this, the thing isn't there. Or if it is, there's no way of determining whether it really is or isn't, so basically it isn't!
From this point on the debate should run pretty analogous to interpretations of quantum theory.
I'd say it is there and it is not there. Both is true. You can't pull it out (not there), but the actual noise would still be different if it hadn't been there. You cannot mathematically determine a point, where all correlation would have vanished, it never does completely. Capture a dirty old shellac disc with 24 bit resolution, with what you consider 30 dB of analog SNR and you still loose the fraction of a bit of
signal, when you decimate that to 16 bit. We don't care, for a reason, but the question is better answered on the grounds of psycho-acoustics or practicability than math.
From this point on the debate should run pretty analogous to interpretations of quantum theory.
The uncertainty principle is from quantum theory. (http://en.wikipedia.org/wiki/Uncertainty_principle)
I'm not entirely sure I should be drawing an analogy here, but it's close.
I'd say it is there and it is not there. Both is true. You can't pull it out (not there), but the actual noise would still be different if it hadn't been there. You cannot mathematically determine a point, where all correlation would have vanished, it never does completely. Capture a dirty old shellac disc with 24 bit resolution, with what you consider 30 dB of analog SNR and you still loose the fraction of a bit of signal, when you decimate that to 16 bit. We don't care, for a reason, but the question is better answered on the grounds of psycho-acoustics or practicability than math.
The re-quantisation measurably increases the noise floor. It's a small fraction of a dB when the original noise floor was already so (relatively) high, but it's there.
I
think it reduces to this: was there (or could there have been) some detectable bit of signal sitting
just above that original noise floor, that has now been swamped by the increase in the noise floor?
That depends on how you try to find the signal.
And whether you separately care about the noise floor.
As you say, psychoacoustics must come into a
sensible discussion - but that's not the point I'm trying to track. In some signals, you realistically risk hiding some signal elements in a way that's trivial to demonstrate, whereas in others, you'd need a steady-state test signal and an hour long FFT to discover any difference in signal detection pre vs post quantisation. If we pick a sensible limit for the temporal integration time, we can calculate what amplitude of steady-state signal can be pulled out of the noise over that period of time. There's a trade-off between the two, and it does start to sound very Heisenberg-ish.
Brain very tired now!
Cheers,
David.
But since it is a lossy codecs then by _definition_ "something" is removed. Otherwise it wouldn't be a "lossy codecs".
Of course, and I expect that point is obvious to all on HA. But to the public at large, who have been told that even low-bit-rate satellite radio is "CD quality," the difference between lossless and lossy bit-rate reduction is not as clear as you might expect. To judge by some of the emails I receive, that difference isn't even as clear as I would expect among Stereophile's readership. That was the point I was making.
If what is removed cannot be discerned by the listener then so far as the listener is concerned the result is still of "CD" quality and it is perfetly reasonable to call it that. The difference may be real, but if it is inaudible then it is irrelevant. "A difference that makes no difference IS no difference".
I think it reduces to this:
Assume a payload signal S1 and its innate noise floor, or uncertainty, N1.
When S1+N1 is properly quantised at a nominal resolution of n bits, this process
introduces a new uncertainty N2(n).
What is the minimal value of n that ensures that N1 > N2, '>' according to a
specific criterion?
The criterion follows from the payload signal
and the method of observation, e.g. is this about humans listening to
music, or is it about correlators digging for an a priori known signal.
This is not important as long as a criterion can be defined.
This is what TBD is trying to establish, working back from a given
recording with n=24.
No MSBs, DACs, monotonicity, or myths need to be mentioned or harmed in this discussion.
I think that puts what I was trying to say in response to googlebot very succinctly.
It would be quite true to argue there are far more relevant/practical ways to look at this (I think Arny keeps trying to take us to correctly dithered signals and their noise floor - which is eminently sensible) - but I'm interested in those LSBs, what's in there, and what happens when they're chopped off (no dither).
I'm interested in the idea that sometimes some of them carry real information, while other times none of them carry any real information at all. I'm interested because this looks like a nice simple idea, while in fact it's rather complicated and difficult to pin down.
It's tenuously relevant if you're going to use noise-shaping and dither to convert to 16 bits, but it's highly relevant in the context of something like lossyWAV, or in considering the ill defined concept of "self dithering".
Anyway, here's a far simpler question: has anyone got a real-world 24-bit recording which has audibly signal-correlated information in the 8 LSBs?
Cheers,
David.
has anyone got a real-world 24-bit recording which has audibly signal-correlated information in the 8 LSBs?
Cheers,
David.
Might be worth having a look here: http://www.2l.no/hires/index.html (http://www.2l.no/hires/index.html)
I've tried a few but nothing so far...
Cheers,
Rob
Anyway, here's a far simpler question: has anyone got a real-world 24-bit recording which has audibly signal-correlated information in the 8 LSBs?
This is going to be tough. We know that any real recording has a noise floor higher than -96 dB so your 8 LSBs are going to have full-scale noise. Best case, there will also be full-scale distorted (amplitude aliased) signal. This gives us some hope. You can get a rough idea of how much hope by listening to the middle 8 bits. Your report on that is that sometimes you can make something out. So the relevant assessment is, do you think you could still make something out if you mixed that with white noise at the same (full-scale) level?
Anyway, here's a far simpler question: has anyone got a real-world 24-bit recording which has audibly signal-correlated information in the 8 LSBs?
This is going to be tough. We know that any real recording has a noise floor higher than -96 dB so your 8 LSBs are going to have full-scale noise. Best case, there will also be full-scale distorted (amplitude aliased) signal. This gives us some hope. You can get a rough idea of how much hope by listening to the middle 8 bits. Your report on that is that sometimes you can make something out. So the relevant assessment is, do you think you could still make something out if you mixed that with white noise at the same (full-scale) level?
What are the current constraints on "real world"?
So the relevant assessment is, do you think you could still make something out if you mixed that with white noise at the same (full-scale) level?
I think for a fair comparison you'd have to mix it before extracting those bits. Then it would act to dither the truncation. So you shouldn't have any audible difference, assuming it dithers it well enough.
I tried this...
a) 16-bit recording
b) truncate a to 8-bits
c) calculate a-b which gives the 8 LSBs of a.
The result is still often white noise, but it's easy enough to find real recordings where it isn't - i.e. where you can hear something related to the original signal.
Using such a recording, I carried out the above steps, and also did
d) add -40dB of white noise to a (i.e. somewhat above the level of the bits that will be removed)
e) truncate d to 8-bits
f) calculate d-e which gives the 8 LSBs of d
c has audible remnants of the original signal, f has nothing but white noise.
Which is what you'd expect - it's basic dither theory. If you don't add enough noise (e.g. -48dB peak-to-peak isn't enough), then some signal correlated elements can be heard in f.
To state the bleeding obvious, there's a huge audible difference between a and b, a and d, and a smaller but still obvious difference between d and e. In all cases the noise level is raised.
So, chopping off the LSBs caused an audible difference
both in the case where those LSBs contained audible signal-correlated content,
and in the case where they contained only white noise. Which again, is as expected, because you need about 5-bits to code white noise without changing the "sound" of it (i.e. to keep the quantisation noise below the noise "floor"), and in this case, it was only being given 1 or 2.
FWIW lossyWAV gets it right, and chops off bits (when possible), always without creating an audible difference.
Coming back to the beginning...
This is going to be tough. We know that any real recording has a noise floor higher than -96 dB so your 8 LSBs are going to have full-scale noise.
I'm not convinced that's always true. I know it's tough to get low real world noise floors across the entire audible band, but it is less tough to get fairly low noise levels at mid-frequencies. Either way, do this while recording some event that's briefly extremely loud, using state of the art equipment, and you might be there. I don't know - a cannon in an anechoic chamber? Realistically, a loud instrument, recorded in a room with 20dB SPL noise or less? That'll get you 100dB+ SNR. You can easily cheat by putting the microphone very close to the instrument, and having only one instrument.
It doesn't sound like many real world recordings to me, but it might have turned up somewhere.
Cheers,
David.
What are the current constraints on "real world"?
Assume you are recording a source with non-lethal SPL (<120 dB SPL signal) in a real room (> 20 dB SPL noise floor).
(what 2BD said)
The 5-bit rule of thumb is new to me. Do you have a reference or can you elaborate?
Based on all of this, it seems like it might be possible to make a case that 24-bit resolution allows a recording to accurately reproduce the character of the noise floor of the equipment/environment in which the recording was made. Your experiments generally support the idea that it is unlikely there is any usable/audible information in those lower 8 bits related to the signal being recorded.
In this 24bit discussion linked away from Slimdevices to "computeraudiophile" someone talks about hard evidence samples and by accident sells these!
http://www.computeraudiophile.com/content/...y#comment-72930 (http://www.computeraudiophile.com/content/debate-24-bit-it-just-audiophoolery#comment-72930)
http://www.soundkeeperrecordings.com/format.htm (http://www.soundkeeperrecordings.com/format.htm)
Anyone examined that bingo-bongo sounds?
In this 24bit discussion linked away from Slimdevices to "computeraudiophile" someone talks about hard evidence samples and by accident sells these!
http://www.computeraudiophile.com/content/...y#comment-72930 (http://www.computeraudiophile.com/content/debate-24-bit-it-just-audiophoolery#comment-72930)
http://www.soundkeeperrecordings.com/format.htm (http://www.soundkeeperrecordings.com/format.htm)
Anyone examined that bingo-bongo sounds?
Do you mean "Kote Moun Yo?" from Equinox? Yes, just white noise in the last 8 bits.
"Dragon Boats" from Lift is far more interesting. Very strange stuff in the last 8 bits. Highly signal correlated. Very little noise. "But that's exactly what you're looking for David". Yeah, it would be if I was sure it was real. Problem is, there are strings of samples (12 here, 50 there, etc etc) scattered throughout this 24-bit file that are already quantised to 16-bits (not to mention the entirely 16-bit fades at either end). It's like the LSBs in this "24-bit" ADC were "sticky" - because, trust me, the signal from a microphone doesn't neatly quantise itself to 16-bits for 0.5ms - so something strange is happening here.
I do have the real "high resolution" disc purchased from that website. I keep meaning to check it (it might just be the upload that's "strange"), but I packed it away to move house nearly a year ago - and I'm still waiting to move!
EDIT: Here's the distribution of bit usage for the 8 LSBs of sr001-01-2496.wav (the downloaded extract of "Dragon Boats" from Lift, above)
[attachment=6397:sr001_01...t15_dist.gif]
I even removed the first second and last five seconds so as not to include the parts that have obviously been processed for upload (i.e. the fade out), though it made barely any difference to the graph because that's such a comparatively small section of the total length.
What you have in that graph is a count of how many times each possible value was found in the file. See the problem? Apart from during virtual silence (there's none in this extract) there's no reason why
any specific value should appear more often than any other. But here, it's the values near 00000000 and 11111111 (255 in decimal) that occur far far more often than those in the middle (~128 decimal). A 24-bit value with 00000000 in the LSBs is really a 16-bit value. What this graph shows is that 24-bit values which are
near 16-bit values are far more likely to occur that those which are midway between exact 16-bit values.
"Proper" 24-bit audio isn't like this. Something is wrong.
It's like sending someone out with a measuring tape to measure sticks, asking them to measure them to the nearest cm, but getting these results...
10
40
29
71
80
1
0
39
11
99
...you've got to wonder why the last digit is always 0, 1, or 9 - couldn't the person read numbers ending in 2-8?!
AFAICT the same thing is happening here with this supposedly 24-bit audio: The ADC seems to struggle measuring values that aren't near to 16-bit ones.
I have no idea how this happened. A best guess would be it's not the ADC at all, but some subsequent processing stage that's gone wrong. However, the recording engineer claims not to do any processing, even normalisation, because it degrades the sound.
EDIT2:
Here's the same graph for sr002-01-24192.wav ("Kote Moun Yo?" from Equinox, above)...
[attachment=6398:sr002_01_24192dist.gif]
There's nothing wrong here - all possible 8LSB values are used approximately the same number of times, exactly as you'd expect.
Cheers,
David.
P.S. I forgot to say: if you check the distribution of sample values for the full 24-bits of sr001-01-2496.wav, you get the typical double-sided poison-like distribution, but with _huge_ spikes at and around the 16-bit values. Here's an overall view...
[attachment=6399:sr001_01...ist_full.gif]
...and here's a zoom in showing the spikes clearly...
[attachment=6400:sr001_01...ist_zoom.gif]
The 5-bit rule of thumb is new to me. Do you have a reference or can you elaborate?
It's something I heard Bob Stuart say once. It seems to be about right, and it makes some sense from a typical simple minimum audible difference of ~ 1dB. I reckon it's closer to 0.5dB and depends on how you measure the noise, but it's something like that.
Sorry, no reference.
Cheers,
David.
"Dragon Boats" from Lift is far more interesting. Very strange stuff in the last 8 bits. Highly signal correlated. Very little noise. "But that's exactly what you're looking for David". Yeah, it would be if I was sure it was real. Problem is, there are strings of samples (12 here, 50 there, etc etc) scattered throughout this 24-bit file that are already quantised to 16-bits (not to mention the entirely 16-bit fades at either end). It's like the LSBs in this "24-bit" ADC were "sticky" - because, trust me, the signal from a microphone doesn't neatly quantise itself to 16-bits for 0.5ms - so something strange is happening here.
I just tried that piece of music and me as noob also wonders that it fades in and out very quick so you don´t have a chance to hear recorded slience. Since my player only plays 96kHz natively i tried 2 soxed versions with 44.1 and 96kHz on my main system. I was even trying listening as loud as my system can but only found the 44.1 version the same sounding.
Edit: I tried both samples offered on their sample page
"Dragon Boats" from Lift is far more interesting.
Possibly a 16-bit recording extended to 24 bits with the wrong dither settings.
Another possibility is High Definition Compatible Digital (http://en.wikipedia.org/wiki/HDCD). This looks like it would introduce some interesting non-linearities in the LS bits. Doesn't quite look like what's going on here though.
The 5-bit rule of thumb is new to me. Do you have a reference or can you elaborate?
It's something I heard Bob Stuart say once.
That seems like counter-evidence to me. Look at all the inaudible *necessities* that Stuart has tried to foist off on the audio world. What do you think you'd get if you asked Stuart to justify his engineering with DBTs?
It seems to be about right, and it makes some sense from a typical simple minimum audible difference of ~ 1dB. I reckon it's closer to 0.5dB and depends on how you measure the noise, but it's something like that.
The myth here seems to be that the number of distinct levels that can be coded is limited by the number of bits.
The true answer is that number of bits required to transparently encode noise has a lot to do with the characteristics of the noise itself and the other rules of encoding. Remember that PWM is essentially 1-bit encoding, and can encode a wide range of levels.
Here we go again, Arny thinks he's spotted another myth to dispel.
For white noise, some specific PDF, 44.1kHz sampling, peak normalised, there must be some specific number of bits at which you can quantise that noise that makes no audible difference, where if you use one less bit, it becomes ABXable.
The clue that I wasn't staking my life on the reliability of the "5-bit" number was in the word that I preceded it with: "about".
Cheers,
David.
Here we have another professional that offers some clearly discernable Hires files...
http://www.whatsbestforum.com/showthread.p...ng-Hi-rez-music (http://www.whatsbestforum.com/showthread.php?2938-The-Art-of-Listening-Hi-rez-music)
There was linked to from a thread at Slimdevices as prove for 24bit superiority. Since the poster claims you feel tones above 18kHz the same way you feel a 12Hz dinosaur stomp i wonder already
lossywav wants to keep a maximum of 16bit at some samples at least but i may misunderstood the handling of its behaviour here!?
The online seller 7digital that offers the latest Radiohead album advertises the 24bit download with using this wording:
"The 24-bit audio resolution is so high it surpasses the signal-to-noise ratio of most consumer electronics. This however means that the sound is transparent and free of low-level digital noise."
Thanks spoon for linking there.
Since we have several samples now that show below there is mostly noise this argument isn´t to much plausible to me anymore. It may be better to have nothing in the last 8 bit, thus 16bit.
Nothing can produce less low-level noise as non existing noise
They may of cause mean there is no need for dithering (noise) such music. Even if its origin is from a higher samplerate and therefore should be dithered when prepared for 24bit/44.1khz imho.
I don´t want to start the same endless discussion like after the apple announcement going 24bit but what do you think?
I only wondered about the way they reasoning the 24bit format. They could have simply written "Blacker Blacks"
I don't think you should spend even one second of your life pondering what a marketing department really meant - after all, they won't have done!
Cheers,
David.