Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: "Audiophile" listening event @ Definitive Audio in Seattle (Read 151552 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #100
Is this even worth debating about?

I'd say "no". Why do we care what John Atkinson can or can't hear?


IMO John throws a gigantic monkey wrench into our presumably shared search for true audio excellence .......

Well, I'm not searching for true audio excellence. I'm quite happy with what I've got.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #101
See my write-up on Philip Hobbs' identical presentation at a 2007 AES Conference:  http://www.stereophile.com/content/watching-detectives .

This report says that the lowest resolution material was 192 kbit MP3 and then later mentions 128 kbit. Your recent demonstrations are apparently 128 kbit. Is that first number an error or one-off experiment or have you done this demonstration successfully and consistently with 192 kbit MP3?

Because of the way the experiment/demonstration is run, the best we can hope to get out of it is a conclusion that the lowest resolution sample played is not transparent. If you're using a format/codec there that has already been demonstrated to be non-transparent, this result is, at best, uninteresting.

Regardless of how gracefully it is reported, it would be reckless to accept anyone's post-listening claim that they started hearing degradation after the first sample.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #102
So, not anything he claims is impossible to demonstrate, only those particular examples. He could claim, hi-res audio has higher resolution than low-res audio, thus making googlebot assumption wrong.

Let's keep with the facts at hand please; you're moving the goal posts.

So then it is your assertion that all other forms of those files sound degraded, or at least different?
any answer to your question would necessarily involve a ToS#8 violation.

googlebot's response was the logical conclusion to the claim; there is nothing wrong or untrue about it.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #103
I don't care what John can't hear, but I do care about what he *can* hear perchance that advances our understanding of the capabilities of the human ear.

IMO John throws a gigantic monkey wrench into our presumably shared search for true audio excellence by relying on only the most flawed and superficial kinds of listening tests that are known to us.

The audiophile is a challenging creature to study; They're elusive and highly intelligent. The endeavor feels something like a Roadrunner cartoon.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #104
That discussion goes well beyond the purview of this thread, I believe.

Your evasion is quite telling, however; at least to me.

Scientific method requires repeatability.  Since you let the cat out of the bag at the end, there is no hope for your experiment.  Furthermore asking people if they heard degradation after it was done does not ensure objectivity.  How many people will say they did because they were concerned about what others thought?  How many people believe that they heard differences after the fact because it was told to them that differences were there?

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #105
That discussion goes well beyond the purview of this thread, I believe.

Your evasion is quite telling, however; at least to me.


I have no desire to get into a debate on the merits or lack thereof of forced-choice ABX testing. All that I think needs to be said is that I acknowledge the subject is a core value of this forum.

Quote
Scientific method requires repeatability.  Since you let the cat out of the bag at the end, there is no hope for your experiment.  Furthermore asking people if they heard degradation after it was done does not ensure objectivity.  How many people will say they did because they were concerned about what others thought?  How many people believe that they heard differences after the fact because it was told to them that differences were there?


I have no idea. As I didn't tally the results, your point is moot.

John Atkinson
Editor, Stereophile

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #106
I have no idea. As I didn't tally the results, your point is moot.

Then I guess your involvement in this discussion is done.  I need only go back to my very first post criticizing your deeply flawed demonstration.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #107
I must agree with googlebot's latest post. Even if JA isn't enamoured by the concept or procedure of double-blind testing, it's hardly so difficult that he couldn't, by now, have done a few quick tests to support his assertions and get them pesky crusadin' objectivists off his back. Excuse the cynic in me for wondering if perhaps he has a vested interest in ignorance (read: comforting subjectivism).


My recollection is that JA has in the past used his perceptions of the difficulty of doing ABX tests as an excuse not to do them. At the time he inferred that only people with deep pockets like AT&T labs and Harman International could afford to to them.

Of course what about those *megabuck*audiophiles from Michigan who call themselves SMWTMS and originated ABX tests?  We must all be trust fund babies! LOL!


"Audiophile" listening event @ Definitive Audio in Seattle

Reply #109
Apologies - I thought I had read it all (even that previous 31 page thread), but apparently read over this.


Thanks.

Quote
See my write-up on Philip Hobbs' identical presentation at a 2007 AES Conference:  http://www.stereophile.com/content/watching-detectives
A nice write up. Which makes it even more bizarre that almost no one successfully ABXes this stuff.


This has been the subject of much discussion on a private list to which I subscribe, which includes JJ, Malcolm Hawksford, Bruno Putzeys, Peter Craven, Bob Katz, Vicky Melchior and other engineers. If there are real benefits to recording at sample rates greater than 48kHz, then why is it difficult to design an ABX test that readily reveals those benefits? One answer to that question, of course, is that there are no benefits, as was recently suggested by a poster to this thread. But personally I feel that it cannot be automatically assumed that absence of evidence is equivalent to evidence of absence.

Quote
I do like the sound (no pun intended) of an unfiltered 78rpm shellac disc as a test signal. A continuous stream of wide bandwidth impulses superimposed over audibly band limited musical content, with the former generally sitting in the middle of the "stereo" sound stage, and the latter (if tracked with a stereo cartridge) flung to all extremes of the sound stage. It kills much lossy coding. It visibly makes anti-alias filters ring like mad. But I can't hear any problems with the latter.


I have ABX'd the differences between the ringing of different low-pass filters. This was in connection with an article by Keith Howard that I published 5 years ago on the subject of reconstruction filters: http://www.stereophile.com/reference/106ringing/index.html . Keith sent me a DVD-A with unidentified music samples on it. _Very_ hard to hear any difference blind, if  at all, though one  thing I did note was that, after the test was over and the examples had been identified, the maximum-phase filter, where all the ringing occurs _before_ the transient, did appear to be audible under double-blind conditions. My apologies for not recalling my score.

John Atkinson
Editor, Stereophile

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #110
Because of the way the experiment/demonstration is run, the best we can hope to get out of it is a conclusion that the lowest resolution sample played is not transparent. If you're using a format/codec there that has already been demonstrated to be non-transparent, this result is, at best, uninteresting.


If the people present had not heard mp3 artifacts before and did at the demo, then I'd guess it was interesting to them. 
Proving their experience as valid to anyone else might be uninteresting

If they knowingly acknowledged differences due to peer pressure, they might adopt an defensive attitude to definitive testing out of fear of not measuring up.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #111
I have no idea. As I didn't tally the results, your point is moot.

Then I guess your involvement in this discussion is done.  I need only go back to my very first post criticizing your deeply flawed demonstration.


With respect, no-one has yet offered any reason why it must have been "deeply flawed." I have addressed the substantive issues that were raised by others, and explained why they weren't relevant.  I went back to your first two posts to this thread (which, please note, I did not start) and this is what you wrote:

From Message #16
Quote
I object to any presentation of lossy encoding that doesn't properly describe the mechanism by which it works and demonstrate how one goes about properly determining whether it works or does not work.

So far you have really only demonstrated to the objective-minded audio community that you are willfully ignorant of lossy encoding, unwilling to improve upon what is a clear deficiency in your presentation of the subject matter as pointed out by people who clearly know more about the subject than you do, and that you are knowingly misleading people. Personally, I find this quite shameful if not morally reprehensible.

Unless you plan on changing your behavior, John Atkinson, offering apologies for being offensive is nothing short of insincere.


From Message #19
Quote
30 minutes is more than enough time to present lossy encoding to the layman audio enthusiast in an honest way. Perhaps it's not enough time if you also want to satisfy your agenda convincingly which still appears to be at odds with an honest presentation.


I see strongly expressed opinion and objections, but no actual criticism.

John Atkinson
Editor, Stereophile

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #112
Ok, if I have been so unfair in assuming fault in your demonstration, feel free to tell us how it is worthy of our* praise.

(*) my use of our is intentional.  While I do not pretend to speak for other members of this forum, it seems clear to you, at the very least, our core principle about how one is to demonstrate differences in sound quality, which is the very purpose of your demonstration.

Let's be crystal clear that the topic of discussion, as put forth by the original poster, is that you had planned a presentation demonstrating the "evils of mp3" and this presentation was likely devoid of an effort to ensure objectivity.  I really should have It really said ABX, but I thought I'd reword it more broadly in order to give you the benefit of the doubt.  As such, don't be surprised to experience some push-back as you continue pretend that you are helping people see the "truth".  That said, I do appreciate that your recent response to David, not withstanding the desperate presentation of the Pink Elephant Orbiting Uranus argument.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #113
Keith sent me a DVD-A with unidentified music samples on it. _Very_ hard to hear any difference blind, if  at all, though one  thing I did note was that, after the test was over and the examples had been identified, the maximum-phase filter, where all the ringing occurs _before_ the transient, did appear to be audible under double-blind conditions.


Well, congratulations for being able to identify the type that's nobody using exactly because of that.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #114
<snip>
I do appreciate that your recent response to David, not withstanding the desperate presentation of the Pink Elephant Orbiting Uranus argument.


Thank you (though I thought it was the Flying Spaghetti Monster argument). It strikes me that the ringing of the different low-pass filters might be an interesting project for HA members. I still have the DVD-A with the  filter examples. If Keith Howard doesn't mind, I could mail it to someone who could then rip the files and make them available to HA members so that they could ABX each type of filter against the original with Foobar. But I am not sure what would be the best way to send the key. (The tracks on the disc are not identified, of course.)

John Atkinson
Editor, Stereophile

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #115
Yay, let's ABX maximum phase filters, that have absolutely no place in audio (except maybe synthesizers and special effects), against those that we already know to be transparent!

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #116
Yay, let's ABX maximum phase filters, that have absolutely no place in audio (except maybe synthesizers and special effects), against those that we already know to be transparent!


I don't understand why you are being sarcastic. I thought that this would be something that would be appreciated on HA and I could publish the results as an addendum to Keith Howard's article. There are 7 different low-pass filters described in the article  - http://www.stereophile.com/content/ringing...s-filter-page-2 - and used to prepare the musical examples on the DVD-A I mentioned. But if posters don't think this would be something they would interested in, then no harm, no foul.

John Atkinson
Editor, Stereophile

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #117
While I don't want to stand in the way of actual testing, in my opinion the presented issues are just resolved. The ear is more sensitive to pre- than post-ringing, so minimum-phase or intermediate-phase beat linear-phase (maximum phase usually isn't even discussed). Then there is the trade-off between filter steepness and ringing. If processing resources are scarce, a gentle roll-off might be preferable. But processing resources aren't scarce since at least a decade. With enough processing power (we talk about 0.1% vs. 0.2% of a modern CPU) I can optimize all low-pass parameters without negatively affecting any others. What's typically available to end users for free, for example the SRC in Foobar or SoX, can deliver so much headroom in every respect, that I just don't see the potential for actual distinction in real world tests.

I also just don't appreciate to see promoted, that questions like these would be in any way practically relevant to modern, high end music reproduction. Just moving the head 2 mm while listening has probably a hundred times more effect on a signal than anything mentioned in your linked article. And these gross non-linearities should be in the center of attention. What can I do to improve the fidelity at an actual listening position and how can I better reproduce an original sound stage in a room of completely different size? Certainly not by just buying a Stereophile approved pair of speakers, that supposedly sounded good in another room, and a tube- or other amp.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #118
With respect, no-one has yet offered any reason why it must have been "deeply flawed." I have addressed the substantive issues that were raised by others, and explained why they weren't relevant.

The flaws in the degraded resolution demonstration: 1/ The assumption that because the subjects can hear a difference between 88.2 vs. 128 MP3, they can hear a difference the comparison between 88.2 and 44.1 that is presented inseparably in the same test. 2/ The test is conducted as a group and responses are not confidential. 3/ The test is single, not double-blind.

Thess flaws have been described in posts 57, 93 and 102 and have not been substantively addressed.

I understand if for whatever reasons you do not consider these to be flaws and believe this is the best and proper way to conduct these evaluations. I'm here to tell you that science has already been down this road with medicine and other formerly subjective disciplines. If these methods were subjected to scientific peer review, it is quite clear they'd be thrown out promptly.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #119
Yay, let's ABX maximum phase filters, that have absolutely no place in audio (except maybe synthesizers and special effects), against those that we already know to be transparent!


I don't understand why you are being sarcastic.
No, me neither. The basic reason we think 44.1kHz sampling should be transparent is because the action of the filter, at inaudible frequencies, should be inaudible. If a maximum phase filter is audible, then we have a problem with the theory. It doesn't matter whether anyone uses such filters or not - it raises the possibility of filters having a "sound" (even if flat/"perfect" within the audible band), which is enough to get reasonable people worried.

I'm sure I read that article (or one very like it) in print in the UK - probably in HiFi news. I've read the filter design section very carefully, and it seems to me that trusting an unwindowed inverse FFT to tell you the truth when you have been tweaking the coefficients is a serious problem. The FFT assumes a periodic signal - it calculates the spectrum for the signal it sees repeated forever. Hence the results can be misleading. Hence we don't really know that Keith's filters were flat below 20kHz. This isn't an idle fear - the exact same issue has bitten me in the past.

There have been threads here on HA attempting to ABX 20kHz LPF maximum phase filters, correctly generated, and we've failed. I think a successful ABX of such a thing would be significant. But the key is that the filters have to be trusted. Not because what happens at 20kHZ should matter, but what happens below 20kHz does matter - so when we're sure that everything below 20kHz is fine, then and only then can any audible problems be attributed to what's happening at/above 20kHz (and possibly other equipment in the chain interacting with what's up there).

Cheers,
David.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #120
Why is it well established then, that minimum or intermediate phase filter designs are preferable over linear phase (in terms of perceived quality), when they must all sound equal to not fall into "a problem with the theory"?

Let alone maximum phase. We know how sensible the ear is to pre-ringing.

Have you got a link to where this test should have failed? I think with the right samples, for example pulse trains with steep attacks, it should be possible to distinguish maximum vs. minimum phase.

The basic reason we think 44.1kHz sampling should be transparent is because the action of the filter, at inaudible frequencies, should be inaudible. If a maximum phase filter is audible, then we have a problem with the theory.


Why must is be that black and white? We believe that 44.1 kHz can be transparent because we can design filters that make it transparent. So finding a filter, that doesn't, doesn't invalidate any theory.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #121
I like the Orwellian "forced choice" description.  I think there's a political party in the US who could use JA's "talents".


Just one?

Just in the US?

;-)

It can be argued that John's demonizing of the process of encouraging choices is counter-productive to the interests of his subscribers and advertisers. If audiophiles make no choices then they buy no equipment, right?

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #122
Yay, let's ABX maximum phase filters, that have absolutely no place in audio (except maybe synthesizers and special effects), against those that we already know to be transparent!


I don't understand why you are being sarcastic. I thought that this would be something that would be appreciated on HA and I could publish the results as an addendum to Keith Howard's article. There are 7 different low-pass filters described in the article  - http://www.stereophile.com/content/ringing...s-filter-page-2 - and used to prepare the musical examples on the DVD-A I mentioned. But if posters don't think this would be something they would interested in, then no harm, no foul.


As others have pointed out, the harm and foul is the implicit claim that the filters evaluated in the "Ringing False" article were relevant to  digital audio in 2006, when it was written.  The methodology used and the choices made are unfortunately characteristic of Stereophile, which is to say woefully out-of-date and irrelevant to audio as being currently practiced in the real world.

Due to the unfortunate choices made by the author, the "Ringing False" article is so fundamentally flawed as to merit being called a straw man. Let's face it, when Stereophile is characterizing digital audio as being inferior to what is actually being delivered in $50 portable music players and the sub-$1 audio interface chips that come *free* on cheap PC motherboards, there is definitely something badly awry in Stereophile land.

It looks to me like a ploy for raising false concerns about digital audio in the interest of selling overpriced high end equipment with zero audible benefits or even deficit performance, floobydust nostrums,  and "Analog Audio" panaceas.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #123
Why is it well established then, that minimum or intermediate phase filter designs are preferable over linear phase (in terms of perceived quality), when they must all sound equal to not fall into "a problem with the theory"?
"well established"? I haven't seen any tests that would satisfy HA rules that show there's any audible difference. We're talking about 20kHz+ here, not in the audible range. Is that what you're referring to? I've no argument with you about the audible range.

Quote
Let alone maximum phase. We know how sensible the ear is to pre-ringing.
In the audible range, yes. What mechanism do you propose for the ear being sensitive to pre-ringing at frequency which are inaudible? (There are two or three possibilities, mostly to do with equipment, and they probably add together to explain the vanishing but very rarely ABXable differences experienced in blind tests).

Quote
Have you got a link to where this test should have failed? I think with the right samples, for example pulse trains with steep attacks, it should be possible to distinguish maximum vs. minimum phase.
We've had exactly that example on HA before - both in the audible range, and at 20kHz...
http://www.hydrogenaudio.org/forums/index....st&p=735236

Quote
The basic reason we think 44.1kHz sampling should be transparent is because the action of the filter, at inaudible frequencies, should be inaudible. If a maximum phase filter is audible, then we have a problem with the theory.

Why must is be that black and white? We believe that 44.1 kHz can be transparent because we can design filters that make it transparent. So finding a filter, that doesn't, doesn't invalidate any theory.
What you say is true, but it's more subtle than that. There are things we think should make a difference (i.e. things within the audible band), and things that we think should not make a difference (i.e. things above the limit of most human hearing - and, FWIW, I suspect way above the limit of JA's hearing, as I believe he's over 60). Obviously we can design filters which aren't ideal within the audible band, causing an audible difference - and as you suggest, that doesn't mean that properly designed filters will cause an audible difference. But if changes above the limit of human hearing also turned out to be audible, then we would have a problem. It may be some cause an audible difference, and as far as we can tell others do not - but that's not really good enough - 1) we don't know why, and 2) we don't know specifically what would constitute a "proper" design, beyond empirical evidence.

Trying to design a "proper" filter, if things we believe to be inaudible turned out to be audible, would be like trying to design a decent lossy codec without any knowledge of masking or how the ear works. If that was where we were at in terms of psychoacoustics, most people wouldn't touch lossy coding with a barge poll.

Any way, this is all nice discussion... but either this can be ABXed, or it can't.

FWIW IMO there's no great harm in recording at a higher sample rate - the problem is when people claim night and day differences, or hail it as the saviour of audio, or claim they actually understand what's happening.

Cheers,
David.

"Audiophile" listening event @ Definitive Audio in Seattle

Reply #124
I brought myself up to date and you are right, David. What I had forgotten was that the ringing frequency is a function of the cut-off frequency, so >20 kHz for the cases in discussion.

So, it would indeed be an interesting finding, if John Atkinson, who very likely can't hear a continuous, high volume 20 kHz tone, could suddenly hear 20 kHz, if it just faintly preceded other signals (and again not if it trailed them).