Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Audibility of "typical" Digital Filters in a Hi-Fi Playback  (Read 325842 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #375
And let's distinguish between hearing and listening.  My hearing is shot due to age.  I am embarrassed to say that I can't even hear 12 Khz well.

 
Thanks, that might be the most refreshingly honest thing I've seen you say.
So if we are to understand this correctly, the artifacts of low-rez 16/44 you "hear" are not due to hearing, but "listening"?
You "listen" to artifacts >12k, but hear only to 12k, or the artifacts you hear with downsampled master >16/44 exist <12k?
I don't recall the ages of the listeners in the BS test being listed and I don't believe their hearing was tested.

cheers,

AJ
Loudspeaker manufacturer

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #376
I don't recall the ages of the listeners in the BS test being listed and I don't believe their hearing was tested.


"
2.4. Listeners
Eight listeners took part in the test, seven of whom
were male. Most of the listeners were audio engi-
neers, and their ages ranged from 25 to 65. All re-
ported normal hearing, although this was not tested
formally.
"

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #377
I guess others (perhaps c|net, Wired, New Scientist, etc.) will report on the Meridian paper's significance (if it has any significance at all, that is).  Anyone seen any reports?

I don't think so.  None of their reporters will be in a position to understand this research/paper.  This is news for professionals in the industry.  Even in forums like this people are struggling to come to terms with it.

That said, I might write an article for it . 
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #378
I guess others (perhaps c|net, Wired, New Scientist, etc.) will report on the Meridian paper's significance (if it has any significance at all, that is).  Anyone seen any reports?

I don't think so.  None of their reporters will be in a position to understand this research/paper.  This is news for professionals in the industry.  Even in forums like this people are struggling to come to terms with it.


I that that the above statements show a backwards idea of what constitutes good writing.  If the Meridian paper wasn't trying to pander so hard to the golden ears, obfuscate so much questionable science, and sneak so many self-serving commercial messages between its lines, it might have been a lot easier to understand.

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #379
Good morning Xnor.  If you don't mind I am going to respond to this in two parts: the personal remarks you make about me below and the technical one.  The latter is actually quite easy and simple misunderstanding.

I beg our moderator's indulgence that this is on topic seeing how Xnor's post remains and not binned.
No, I started up pretty "deaf."  I remember being shocked that I could not hear much difference between 128 kbps MP3 and the source.  That bothered me .  So I started on a path to become a trained listener and after literally hundreds of hours of testing, and tons of education learning about psychoacoustics and algorithms in audio processing, all of a sudden found myself in a super unique situation of outperforming everyone else around me in such tests.  I have never liked the phrase but everyone would call me the "golden ear."

The experience taught me to use my ears as an instrument and separate it from enjoyment of music.  Today I can't let go of that skill.  It seems to be with me forever.  For that reason US DBS broadcasting (eg. XM) bothers me to no end.  I love the content but can't listen to it.

And let's distinguish between hearing and listening.  My hearing is shot due to age.  I am embarrassed to say that I can't even hear 12 Khz well.  The reason I do well still I think is that I focus on what I think technically matter and combine that with my training to look for small differences.  I suspect people with my training but intact frequency response will be able to do a lot better.  I actually ran into one such person at Microsoft who was working with a partner of ours.  He was hearing high frequency distortions I could not.  We hired him immediately .  And I moved off from doing a lot of the listening tests myself.

Let me confess.  I had no idea I could do this after so many years after retirement.  I just ran the tests because people like Arny kept egging me on so I gave it a try.

Having read the Stuart paper and prior citations, I am starting to think this may be due to time domain impact.  It certainly is not frequency domain since I can't hear the ultrasonics. Just as pre-echo is a very audible time domain artifact (although created in frequency domain), maybe these filters act the same way.


After reading this I get an even stronger sense that you must have been cheating in a lot of the logs you posted, or you have some seriously anomalous hearing (regarding detecting lossy compression).

Let me express a big sigh of relief that you did not call me ugly too!  A man is not a man without his good looks. 

If I may, I like you to step back and think about what you have said/are doing.  Our shtick in these discussions has always been a challenge for the other camp to run an ABX test to prove their subjective impressions of audio.  Since I am not in the opposing camp, and see value in blind tests, I ran the tests.  Not one but many.  Not the first version but the follow ups.  Not just on AVS but also here and on WBF Forum.  What’s has been the response?  What you say: either I have some hearing disease I am too dumb to understand, or fabricated test results.

Imagine what impression this leaves in the minds of anyone who googles about this research and lands on this thread and your post.  The inescapable conclusion would be to never run an ABX/blind test.  Nothing good comes out of it.  If they run it and get negative outcome, it would be bad for their position.  Even worse is actually running them and getting a positive result!  Hell will break loose. The goal post suddenly moves from audio to their own personal integrity.

If our real intentions are to encourage others to believe and run such tests, we have failed and failed badly in that mission.  I mean if the “inventor of ABX” test, our beloved Arny, goes and declare his test of high resolution audio “invalid,” clearly no one is qualified to create a valid one.  At least not one that has any chance of creating a positive outcome.

Now, let’s dig into your theory.  How many such tests have you run?  Can you pass any of them?  I even allow you to cheat and pass them.  Just looking to see any, any indication that believing in blind tests is actually real.  So far I have only seen Arny run one test and he said he cheated.  When I asked him to re-run it with the latest ABX test with hash signatures, he refuses.  Steven (Krab) has not posted any.  AJ has not posted any.  Mzil has done a couple but he insists to have found a cheat and not really heard a proper difference.  I haven’t seen any from our moderator or others arguing with me.

That is a tough place to be. So what to do?  What to do?  Let’s claim impossibility here.  Never mind that we are discussing a published double blind study that was peer reviewed test that found similar outcome.  What do you think?  They are also fabricating the results or have weird hearing?  It can't be that your knowledge of what is or is not audible is incomplete?  You have mastered all there is to know?  Maybe you link us to what professional or academic research has led you to that position.

My wish for you is to put aside your doubts and run the tests.  See if you can focus and find some difference.  You may, just may, develop critical listening abilities.  And start believing in reality of some people being able to way outperform other listeners.  And what they do, is no cheating, or miracle of hearing but like a sport, is due to training.

That is if you care to learn about these experiments.

Amir
Retired Technology Insider
Founder, AudioScienceReview.com

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #380
Even in forums like this people are struggling to come to terms with it.

And you base this on what? Two people losing the actual topic and paper contents out of sight in heated back and forths?
You know, some people don't even own the paper. That doesn't stop people from pointing out potential flaws from the limited information available, or pointing out nonsense written by others.

That said, I might write an article for it .

Doing an interpretation for the audiophile laymen to tell them what they want to hear? 


Btw, you still haven't explained why 21+ kHz filter ringing is of concern for your ears. (Not that I believe that your ABX logs are not based on chance or cheating...).
That seems to be the only argument you have left.

And regarding the paper: there are still unknowns, such as the switching software, so that we cannot draw any reliable conclusions from it.
Stuff like this really should be immediately attempted to be reproduced by some unaffiliated group...
"I hear it when I see it."

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #381
I guess others (perhaps c|net, Wired, New Scientist, etc.) will report on the Meridian paper's significance (if it has any significance at all, that is).  Anyone seen any reports?

I don't think so.  None of their reporters will be in a position to understand this research/paper.  This is news for professionals in the industry.  Even in forums like this people are struggling to come to terms with it.


I that that the above statements show a backwards idea of what constitutes good writing.  If the Meridian paper wasn't trying to pander so hard to the golden ears, obfuscate so much questionable science, and sneak so many self-serving commercial messages between its lines, it might have been a lot easier to understand.

Good morning.  I appreciate your difficulty in getting through it.  Fortunately the members of AES peer review board did not have any such difficulties:



If they could get past all of the things that are bothering you, how come you can't?  They found good learning in it or they would not have bestowed this award on them.  Right?  Why can't you put the resentment aside for a second and let the knowledge and data sink in?  Too difficult?
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #382
Even in forums like this people are struggling to come to terms with it.

And you base this on what? Two people losing the actual topic and paper contents out of sight in heated back and forths?

Two people?  I count a lot more than two people.  But sure, why don't you summarize for us what you think the test was and the outcome.

Quote
Btw, you still haven't explained why 21+ kHz filter ringing is of concern for your ears. (Not that I believe that your ABX logs are not based on chance or cheating...).

As I mentioned, you are confused about that.  I will explain later.

Quote
That seems to be the only argument you have left.

You mean other than a boatload of positive ABX test results?  You mean other than a peer reviewed double blind listening test that is published by AES with authors that are true experts in the industry?  None of that counts but the rhetoric on this forum by constantly declaring the game over in your favor? 

Quote
And regarding the paper: there are still unknowns, such as the switching software, so that we cannot draw any reliable conclusions from it.
Stuff like this really should be immediately attempted to be reproduced by some unaffiliated group...

Unfortunately I don't recall any of you immediately or at any time trying to repeat Meyer and Moran tests.  But now we get religion.  Why?  Because we don't like or understand the outcome.  That aside, here is you saying you don't know the full story still after so many pages.  So why object to me stating the same?
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #383
I don't think so.  None of their reporters will be in a position to understand this research/paper.  This is news for professionals in the industry.  Even in forums like this people are struggling to come to terms with it.

Please  .
It's fodder for the Hi Rez $cam peddlers and the $cam addicts, the only ones buying "Hi-Rez".
They did wisely choose a 2L track with no (Meridian recommended TPDF downsampled) 16/44 version.
.[/quote]
Aimed at industry professional insiders, or $cam addicts who buy this stuff?

cheers,

AJ
Loudspeaker manufacturer

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #384
I don't think so.  None of their reporters will be in a position to understand this research/paper.  This is news for professionals in the industry.  Even in forums like this people are struggling to come to terms with it.

Please  .
It's fodder for the Hi Rez $cam peddlers and the $cam addicts, the only ones buying "Hi-Rez".

Nah.  They don't care about our tests AJ.  It would be like asking an atheist to swear on a bible god doesn't exist.

It is our camp that is crying in anger.  It is kryptonite for people who think small distortions are inaudible.  You remember kryptonite, yes?

It is *our* worst nightmare.  People in our own camp running tests and passing the impossible ones.  As expected, there is a lot of crying.  We had that on AVS but it eventually died down.  The evidence is nothing like we have faced before.  No subjectivist claiming one cable to sound better than other.  No.  The evidence is the bible we believe: passing double blind ABX tests.  We could now turn around say such tests are no good but only makes us look worse.  Much worse.

2014 will go down as the end of an era whether you like it or not.  The era of believing hobbyist tests just because it got published in the Journal.  The era of believing stuff you read online as gospel.  Era of people claiming to be objectivists yet be the most afraid to run double blind ABX tests.  Era of people thinking by reading stuff online they all of a sudden are audio experts.  Era of people using their lay understanding of audio science and confusing it with the real thing.

The storm on this forum shall too pass.  It has to.  It is all a lot of empty protests at the end of the day.  I am here for you AJ.  You can cry on my shoulder but don't expect me to do the same .
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #385
If I may, I like you to step back and think about what you have said/are doing.  Our shtick in these discussions has always been a challenge for the other camp to run an ABX test to prove their subjective impressions of audio.

ABX is just one method that has its flaws. We want to see reliable, valid, objective, reproducible ... evidence for any claim.
Neither of these are given by posting ABX logs, so there is some faith involved in trusting the person and logs.

So what happens is not only skepticism against the original claim, but also the claim that the ABX log is genuine if the result is surprising. (Who cares if someone fudged at 128 kbit mp3 ABX log, right? We know that at this bitrate transparency is not given.)
This leads people to trying to reproduce what the person allegedly heard. Now what if we have many people trying to reproduce the result but failing, and a few that also claim to have heard differences?

These people are either:
- cheating as well
- hearing some artifact of their system that is not related to the actual comparison
- have some anomalous hearing (e.g. psychoacoustic models of lossy encoders expect normal hearing)
...
- have better hearing than everyone else (which you boldly admitted to do not) and genuinely hear a difference

Did I miss something important?


Not one but many.  Not the first version but the follow ups.  Not just on AVS but also here and on WBF Forum.  What’s has been the response?  What you say: either I have some hearing disease I am too dumb to understand, or fabricated test results.

No, I'm not saying that.
I'm saying that in some of these tests, the more incredible ones, you either
- didn't genuinely hear a difference (this includes purely guessing right a couple of times, hearing differences that are due to some incompetent person being unable to filter/resample files properly, abusing some artifacts in your soft- and hardware configuration ...) or
- you have anomalous hearing that can somehow magically hear 21 kHz ringing despite a roll-off above 12 kHz or
- you think you genuinely hear a difference, but which is actually an artifact of one of the problems mentioned above
- you actually hear a difference, throwing over quite a bit of what we know about psychoacoustics, physics, hearing ...


Imagine what impression this leaves in the minds of anyone who googles about this research and lands on this thread and your post.  The inescapable conclusion would be to never run an ABX/blind test.  Nothing good comes out of it.  If they run it and get negative outcome, it would be bad for their position.  Even worse is actually running them and getting a positive result!  Hell will break loose. The goal post suddenly moves from audio to their own personal integrity.

Disillusioning yourself is a negative outcome? If I believe that I can hear a filter far above my hearing range, but fail all tests, I would have been better off not making the test and keeping my false belief? Or even worse, cheating so that the test results conform with my false beliefs? Seriously, wtf?

Again, even if you genuinely get a positive result, there's a chance of it being a false positive. Nobody here accepts ABX logs or the results of any paper blindly, except for trivial stuff. That doesn't mean that ABX logs are never accepted, they often are (see above for reproduction of the results).

Let's try an example:
A guy shows up posting ABX logs, claiming to hear differences between AIFF and WAV files. You look at the files, the audio data is identical, the player decodes them identically ... so you say you don't believe this result is genuine. A few other guys show up posting similar logs. Hundreds of others have also tried, but failed. Do you now believe the claim?


If our real intentions are to encourage others to believe and run such tests, we have failed and failed badly in that mission.  I mean if the “inventor of ABX” test, our beloved Arny, goes and declare his test of high resolution audio “invalid,” clearly no one is qualified to create a valid one.  At least not one that has any chance of creating a positive outcome.

Why? An honest person would look for the reason for an incredible positive ABX result, trying everything to eliminate the possibility of a false positive.

Positive outcomes are easy to produce. The hard part is eliminating false positives.


Now, let’s dig into your theory.  How many such tests have you run?  Can you pass any of them?  I even allow you to cheat and pass them.  Just looking to see any, any indication that believing in blind tests is actually real.  So far I have only seen Arny run one test and he said he cheated.  When I asked him to re-run it with the latest ABX test with hash signatures, he refuses.  Steven (Krab) has not posted any.  AJ has not posted any.  Mzil has done a couple but he insists to have found a cheat and not really heard a proper difference.  I haven’t seen any from our moderator or others arguing with me.

I regularly run ABX tests. I couldn't count them. Yes, I passed many in the past, but have a hard time hearing differences at 21 kHz when my hearing rolls off at 18 kHz.

Have you ever considered the possibility, that people don't post their negative ABX results? For each successful ABX log posted at a public test there are probably tens to hundreds of failed logs that people do not submit.

I won't cheat. If I post a fake log, I clearly state so. I do all the checks I can, to not post false positives for hard tests. I wouldn't want to be seen in the same light as some people see you, who say they trivially pass said tests (which should actually be extremely hard to impossible) and throw logs around.


Never mind that we are discussing a published double blind study that was peer reviewed test that found similar outcome.  What do you think?  They are also fabricating the results or have weird hearing?

Do you understand the difference between someone reading a paper and not spotting gross flaws vs. someone else actually trying to reproduce the results?


It can't be that your knowledge of what is or is not audible is incomplete?  You have mastered all there is to know?  Maybe you link us to what professional or academic research has led you to that position.

You surely would make a "good apologist" for your favorite religion. I hope I don't need to explain the utter nonsense that you just wrote. (Scroll back a few pages, where people have explained the basics of logic and science at least 3 times to you.)


My wish for you is to put aside your doubts and run the tests.  See if you can focus and find some difference.  You may, just may, develop critical listening abilities.  And start believing in reality of some people being able to way outperform other listeners.  And what they do, is no cheating, or miracle of hearing but like a sport, is due to training.

That is if you care to learn about these experiments.

What tests?
Before I can be honest about ABX testing I need files that were created competently. Otherwise I end up with false positives as I did with some of the files that you use to throw out positive ABX logs.

... and no, hearing 21+ kHz is not a matter of training. It's a matter of having pristine, young ears and being blessed with excellent hearing.


@mods: Please extract this into a new thread if it bothers you.
"I hear it when I see it."

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #386
What you say: either I have some hearing disease I am too dumb to understand, or fabricated test results.
Imagine what impression this leaves in the minds of anyone who googles about this research and lands on this thread and your post.

Amir, please don't blame xnor, I made him aware of this thread, so he knows your methods and will expect you to Red Herring the argument into how many tests he/someone else did....

How many such tests have you run?  Can you pass any of them?  I even allow you to cheat and pass them.  Just looking to see any, any indication that believing in blind tests is actually real.  So far I have only seen Arny run one test and he said he cheated.  When I asked him to re-run it with the latest ABX test with hash signatures, he refuses.  Steven (Krab) has not posted any.

Yes, exactly like that!

AJ has not posted any.

I've explained to you multiple times that I have no pecuniary interest in this matter and would not stoop to gaming/cheating on unsupervised Windows online games. Further, your post is false, in the sense that I have partaken in unsupervised Windows online games, but when I found differences due to cheatability/gaming, I did not parade my "victory" around forums. I reported it to the website.
None of which have one iota to do with the BS paper.

cheers,

AJ
Loudspeaker manufacturer

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #387
Stuff like this really should be immediately attempted to be reproduced by some unaffiliated group...

There are way too many vague explanations and details missing for this to be possible.
Switching software and speakers are the tip of the iceberg.

Amir, here is a basic one, at what distance were the listeners from the speakers in this test?

cheers,

AJ
Loudspeaker manufacturer

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #388
Quote
Btw, you still haven't explained why 21+ kHz filter ringing is of concern for your ears. (Not that I believe that your ABX logs are not based on chance or cheating...).

As I mentioned, you are confused about that.  I will explain later.

I'm waiting.

You mean other than a boatload of positive ABX test results?  You mean other than a peer reviewed double blind listening test that is published by AES with authors that are true experts in the industry?  None of that counts but the rhetoric on this forum by constantly declaring the game over in your favor?

You mean the false positive / cheated ABX logs? Well... that was easy.

The paper describes people with better hearing and I guess better hardware (we will never know what system you use to ABX since you don't answer the question posted pages back) and I see this as a separate thing vs. your claims.

I have never claimed game over regarding the paper. Stop strawmanning.
All I did so far are pointing out potential problems, or commented on problems in other people's posts. It is called being skeptical, which I am towards all claims.


Quote
And regarding the paper: there are still unknowns, such as the switching software, so that we cannot draw any reliable conclusions from it.
Stuff like this really should be immediately attempted to be reproduced by some unaffiliated group...

Unfortunately I don't recall any of you immediately or at any time trying to repeat Meyer and Moran tests.  But now we get religion.  Why?  Because we don't like or understand the outcome.  That aside, here is you saying you don't know the full story still after so many pages.  So why object to me stating the same?

Seriously, if I didn't have a headache from your previous posts already, I would bash my head against the wall instead of just facepalming.
Do I really need to give you an introductory course about null and alternative hypotheses, when to reject the null hypothesis, that we never accept it ...? You can't say "Meyer and Moran, therefore no audible difference", so why do you post such a rubbish reply?

It also seems like you are assuming that I'm in one of those two strictly divided camps that you have in your head. Unless you stop that stereotypical thinking I think I've had enough for a while.
"I hear it when I see it."

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #389
The evidence is nothing like we have faced before.  No subjectivist claiming one cable to sound better than other.  No.  The evidence is the bible we believe: passing double blind ABX tests.  We could now turn around say such tests are no good but only makes us look worse.  Much worse.

No, "we" could just report the test "passing" like I did to Klippel.

2014 will go down as the end of an era whether you like it or not.  The era of believing hobbyist tests just because it got published in the Journal.  The era of believing stuff you read online as gospel.  Era of people claiming to be objectivists yet be the most afraid to run double blind ABX tests.  Era of people thinking by reading stuff online they all of a sudden are audio experts.  Era of people using their lay understanding of audio science and confusing it with the real thing.

So they've legalized it in WA too. I had no idea, haven't really been keeping track. It failed an amendment down here.

I am here for you AJ.

I know, I invited you. 
I'll just keep pointing out all the many issues with the BS paper, not worry about the personalities.

cheers,

AJ
Loudspeaker manufacturer

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #390
2014 will go down as the end of an era whether you like it or not.  The era of believing hobbyist tests just because it got published in the Journal.  The era of believing stuff you read online as gospel.  Era of people claiming to be objectivists yet be the most afraid to run double blind ABX tests.  Era of people thinking by reading stuff online they all of a sudden are audio experts.  Era of people using their lay understanding of audio science and confusing it with the real thing.

So they've legalized it in WA too. I had no idea, haven't really been keeping track. It failed an amendment down here.



This is too damn funny.

Sorry, that's the entirety of my contribution.
FLAC -> JDS Labs ODAC/O2 -> Sennheiser HD 650 (equalized)

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #391
I guess others (perhaps c|net, Wired, New Scientist, etc.) will report on the Meridian paper's significance (if it has any significance at all, that is).  Anyone seen any reports?



They may well 'report' on it, but will they be in a position to evaluate its significance to home audio?

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #392
I don't think so.  None of their reporters will be in a position to understand this research/paper.  This is news for professionals in the industry.  Even in forums like this people are struggling to come to terms with it.

Please  .
It's fodder for the Hi Rez $cam peddlers and the $cam addicts, the only ones buying "Hi-Rez".

Nah.  They don't care about our tests AJ.  It would be like asking an atheist to swear on a bible god doesn't exist.



You know,  AJ has posted that Meridian blurb strongly endorsing TPDF dither (as AES gods Lipshitz and Vanderkooy first did long before Meridian) at least a half dozen times now....admittedly only about half as often as you post your fabulous ABX results, but still, you can't have missed it --  and he's requested your thoughts on it every time,  re: the disjunction between it and what was reported in Meridian's convention paper.  Do you plan to ever engage?

(If you do, please try to restrain yourself to maybe just one or two twitter-length paragraphs.  See if you can do that. Consider it a character-building exercise, by character-deleting.)

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #393
No, I'm not saying that.
I'm saying that in some of these tests, the more incredible ones, you either
- didn't genuinely hear a difference (this includes purely guessing right a couple of times, hearing differences that are due to some incompetent person being unable to filter/resample files properly, abusing some artifacts in your soft- and hardware configuration ...) or
- you have anomalous hearing that can somehow magically hear 21 kHz ringing despite a roll-off above 12 kHz or
- you think you genuinely hear a difference, but which is actually an artifact of one of the problems mentioned above
- you actually hear a difference, throwing over quite a bit of what we know about psychoacoustics, physics, hearing ...



Apparently the last one, which I think we'd all call an extraordinary claim,  would not require 'extraordinary evidence', that's just some dumb meme Carl Sagan propagated, we are told.  (Though I would call the Meridian methods extraordinary in some senses,  if not particularly germane to the standard line on hi rez audio)

The excitable pro hi-rez parties (and here I do *not* include the authors of the Stuart paper, at least not so far) don't seem terribly interested in figuring out what their extraordinary results might actually *derive from* , which is what any scientist would focus in on with laser intensity.  Our most excitable party just wants to trumpet it as heralding a New Age in audio hobby discourse.

Hi rez for home audio has been touted since the late 1990s.  We had years of tests -- not just M&M, but also those who attempted to replicate Oohashi et al, and of course here-- indicating only the most modest 'difference' if at all, and not really 'easy' to detect by most people, without some 'enhancement' of the difference cues (hell, in the case of Oohashi et al, it required *not* using ABX and *using* an MRI scanner).  Now suddenly we have a cluster of virtually slam-dunk  'positives' from a handful of online trials, and another modest positive from Meridian's work.  Certain excitable parties (who happen to sell 'hi rez' hardware) find this very exciting. Curious, no? 


Quote
Have you ever considered the possibility, that people don't post their negative ABX results? For each successful ABX log posted at a public test there are probably tens to hundreds of failed logs that people do not submit.



That's one of the 'train wreck' aspects of the AVS Forum trials.  To the test moderators' credit, though, they have asserted multiple times that the test and results there are not scientific.

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #394
What you say: either I have some hearing disease I am too dumb to understand, or fabricated test results.
Imagine what impression this leaves in the minds of anyone who googles about this research and lands on this thread and your post.

Amir, please don't blame xnor, I made him aware of this thread, so he knows your methods and will expect you to Red Herring the argument into how many tests he/someone else did....


The key phrase that one finds in Amir's first post to that thread is:

"I did not level match anything. However, once I found one source was worse than the other, I would then turn up the volume to counter any effect there. Indeed, doing so would close the gap some but it never changed the outcome. Note that the elevated level clearly made that source sound louder than the other. So the advantage was put on the losing side."

This is just one more reason why when Amir says "Our camp" I wonder WTF he thinks he is talking about.

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #395
ABX is just one method that has its flaws.

Them are fighting words for some here! 

Quote
We want to see reliable, valid, objective, reproducible ... evidence for any claim.

For any claim?  How about giving me a link to one that has passed all of those here so that I get calibrated.

Quote
Neither of these are given by posting ABX logs, so there is some faith involved in trusting the person and logs.

Well, do we get to say Meyer and Moran's test reports are not reliable?  Or do we take it on faith that it is because it had negative outcome?  Again, I am trying to get calibrated.

Let's say you don't trust the person.  What then?  He has to get a live witness?  Why on earth would he want to go through all that trouble to convince you?  What is in there for him to have an anonymous poster be satisfied in this regard?

And why do you post under an alias?  Shouldn't I be able to examine your history and who you are in judging your opinions and any test results you put forward?  You have mine.  How come you are afraid of transparency on your part?

Quote
So what happens is not only skepticism against the original claim, but also the claim that the ABX log is genuine if the result is surprising.

Your skepticism is due to lack of experience and knowledge.  You can't turn that around and put the work on me to prove you wrong.  We have peer reviewed tests in the form of Stuart's tests.  We also have peer reviewed tests in the form of Meyer and Moran.  In neither case will I remotely go to the place you are, questioning the integrity of people.

Run these tests yourself.  See if you can pass them. There are people on AVS Forum who shocked themselves after following me lead and carefully listening, found differences reliably.  Until you demonstrate that you have done that, your suspicions are not an issue I worry about.

Quote
(Who cares if someone fudged at 128 kbit mp3 ABX log, right? We know that at this bitrate transparency is not given.)

Oh I would.  If they cheated I wouldn't want to have anything to do with them in anything.  I have little patience for people with lack of integrity. 

Quote
Now what if we have many people trying to reproduce the result but failing, and a few that also claim to have heard differences?

These people are either:
- cheating as well
- hearing some artifact of their system that is not related to the actual comparison
- have some anomalous hearing (e.g. psychoacoustic models of lossy encoders expect normal hearing)
...
- have better hearing than everyone else (which you boldly admitted to do not) and genuinely hear a difference

Did I miss something important?

Of course.  You missed the most important one: training.  Are you really not aware of the concept of trained/expert listeners?  You think anyone who does better has better hearing?  How come with the same hearing ability I went from not hearing compression artifacts to outperforming all but one person in my entire career at Microsoft? 

The other major thing you missed is knowledge of the technology in question.  I know what to listen for.  When I took David's test I quickly zoomed in on the right note and the game was over. If you don't know that, you be lost in the woods when it comes to dynamic distortion.    The difference may be there and audible to that person but if you don't know where it is in a 3 minute song, good luck finding it by randomly clicking here and there.  There is a science to what is relevaing and what is not.

Let me give you a real life example.  The year is 1999 or early 2000.  The music industry is up in arms about piracy.  It forms a consortium called SDMI chaired by Leonardo whom you may know has chairman of MPEG to remedy that situation.  One of the things they went after was adding audio watermark to DVD audio.  They put out a call for proposal and Microsoft Research put one that they were working on forward.  I hear about them doing that just before submission.  I ask them to let me listen to see if the mark is transparent (big deal as far as acceptance by the labels).  They assure me that they had done their listening tests and no one could tell the difference.

I ask them to humor me and give me the tracks which were supplied by the labels.  I don't know how much you know about audio watermarks but the concept is data hiding where the encoder attempts to find segments where masking would easily cover the toggling of the bits.  Back to the story, they files were 24 bit 96 Khz and 3+ minutes long.  I listen and I can't tell the difference.  Bothered , I try harder and all of a sudden I detect something in a fraction of a second.  It was just a transient that didn't sound right to me.  I go back to the Microsoft Research manager and tell him that I thought I had found the difference.  With disbelief, he asks me for the timecode where I heard it.  I give it to him in seconds and milliseconds .  The reaction was golden.  He comes back after a bit and says I had indeed found it and was a bug in the code!  They fixed it and all was well.

This is what training,  critical listening ability and technical knowledge enables.  To people without those attributes, it comes across as something impossible.  But with objective confirmation of the bug in the code, you can take what I said to bank.

By the same token, unless you can demonstrate to me that you are an expert in this area, your incredulity I am afraid has no weight or importance to me.

The above is one of the most important lessons here.  We have proven beyond any doubt that the above skills exist and you cannot take your listening results as a measure of whether someone else can or cannot hear small distortions.  This is why I don't care what Meyer and Moran found.  They did not have critical listeners so their results don't apply to me.

I will stop here and encourage you to go and take these tests.  Try hard to pass them.  You might, just might be able to do that.  If so, you will learn something valuable.
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #396
I'd be curious, Amir, in understanding just what training you undertook while at Microsoft? Myself, I'd prefer to work under the assumption that You didn't cheat. One thing I find just a bit peculiar is that your success at using ABX is at odds with what Stuart et al said regarding that approach, being that it was more difficult. Thoughts?

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #397
The paper describes people with better hearing and I guess better hardware (we will never know what system you use to ABX since you don't answer the question posted pages back) and I see this as a separate thing vs. your claims.

Not it doesn't.  The listeners were not expert/trained listeners.  They received training for the test.  That is a different animal than someone who has spent years becoming an expert listener.  And further, has the technology experience to know what to listen for as I just explained.

Here is another example, this time from the work of current president of AES, Dr. Sean Olive.  Download the software from this link and take the blind test: http://www.whatsbestforum.com/showthread.p...le-For-Download

It is a simple test.  Just a set a parametric EQ whose frequency is randomly picked by the software and the Q progressively increased.  When I first ran the test I couldn't go past level 3 or so.  Then I listened to a bit more and stopped.  A while later I am at Harman and Sean is presenting to a group of us the same tool.  Up to level 3 most people were getting it right.  Past that I was the only one that could keep up with Sean.  I went up to level 6 or 7 and my limited training capped out.  Sean had no trouble going past that.  I think he said the minimum level for their trained listeners is 12.  Try it.  Only then will you get a firsthand feel for what it means to be good at listening tests, trying to find small differences.  And remember, your "hearing" stays constant.  It is your listening skills that improve.

So no, they are not similarly situated to me.

Quote
Quote
And regarding the paper: there are still unknowns, such as the switching software, so that we cannot draw any reliable conclusions from it.
Stuff like this really should be immediately attempted to be reproduced by some unaffiliated group...

Unfortunately I don't recall any of you immediately or at any time trying to repeat Meyer and Moran tests.  But now we get religion.  Why?  Because we don't like or understand the outcome.  That aside, here is you saying you don't know the full story still after so many pages.  So why object to me stating the same?

Seriously, if I didn't have a headache from your previous posts already, I would bash my head against the wall instead of just facepalming.
Do I really need to give you an introductory course about null and alternative hypotheses, when to reject the null hypothesis, that we never accept it ...? You can't say "Meyer and Moran, therefore no audible difference", so why do you post such a rubbish reply?

The only thing rubbish is this uninformed opinion. Null hypothesis?  Are you kidding me?  Meyer and Moran violated just about every rule for properly conducting a listening test.  I am asking you are not pointing those things out if you have genuine intentions. 

Quote
It also seems like you are assuming that I'm in one of those two strictly divided camps that you have in your head. Unless you stop that stereotypical thinking I think I've had enough for a while.

You are showing extreme angst and frustration over these findings, both mine and that of stuart.  You are being accusatory and emotional.  And resentful.  These are characteristics I see in my stereotype of the people you talk about.  In contrast, I don't see that in David. Yes he is disagreeing with me but he is demonstrating calm, knowledge and measured interactions.  So please don't put this at my feet.  Think before you let your emotions write the words if you want to see a different impression of you.  And remember, you are posting under an alias.  So you can't have any expectation of me knowing who you are.

BTW, I would be perfectly fine with you stopping this interaction.  I still owe you that technical answer but otherwise, I am getting tired of dealing with another person's emotional outbursts.
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #398
I'd be curious, Amir, in understanding just what training you undertook while at Microsoft? Myself, I'd prefer to work under the assumption that You didn't cheat. One thing I find just a bit peculiar is that your success at using ABX is at odds with what Stuart et al said regarding that approach, being that it was more difficult. Thoughts?

Hi Chu.  Good to see you here and thanks for chiming in.

Stuart is absolutely right.  And would be even more right had he had to endure taking the test using the foobar ABX plug-in. 

It is an incredibly difficult challenge when one is detecting small differences.  Remembering an image of A in mind, then B and then X when differences are small is very challenging.  With discipline I am able to do it better than most but it requires so much concentration because of the test fixture not being ideally suited to finding small differences.

Ironically placebo or better said, nocebo plays a very strong role here.  You listen to A and then B.  Let's say you hear a valid difference that can be objectively proven.  I don't know about others but I can easily "un-hear" that difference!  What placebo giveth, nocebo taketh away!    I can take fidelity differences that one would value like a beautiful reverb trail and with the right level of doubt, erase it as if it does not exist.

The above is deadly in the tests we are talking about.  I found a work-around but it relies even more on memory.  What I do is listen to A and B and classify their difference.  Once there, I do NOT allow myself to second guess myself when listening to each trial.  If I let myself have any doubt, I get lost and fail the rest of the test.

The poor usability of Foobar ABX plug-in just adds to it.  Try to create a segment using that slider in a 3 minute song.  You will go nuts.  Tiniest movement of the mouse jumps ahead many seconds.  Take the test once and try to take it again and there is no memory of what segments you had picked.  Non-deterministic glitches and popps add to the "excitement."  It is like trying to run a marathon while people throw rocks at you.  The job is difficult enough without the tool getting in the way.

The newest version takes more steps backward.  It seems the goal is to make it harder and harder for people to get positive results.  I used it and passed David's test to deal with people's accusations of cheating but it was even more difficult to use than before.

Now, is the alternative of using sighted tests better?  Not at all.  The magnitude of differences is so small that placebo effect easily dwarfs it.  So we are stuck between two non-ideal methods to get to the "truth."  This is why I like mathematical analysis such as what Stuart does so beautifully.  Or measurements.  We take out the untrustworthy listener out of the picture and get truly objective data.
Amir
Retired Technology Insider
Founder, AudioScienceReview.com

Audibility of "typical" Digital Filters in a Hi-Fi Playback

Reply #399
You are showing extreme angst and frustration over these findings, both mine and that of stuart.  You are being accusatory and emotional.  And resentful.  These are characteristics I see in my stereotype of the people you talk about.  In contrast, I don't see that in David. Yes he is disagreeing with me but he is demonstrating calm, knowledge and measured interactions.  So please don't put this at my feet.  Think before you let your emotions write the words if you want to see a different impression of you.  And remember, you are posting under an alias.  So you can't have any expectation of me knowing who you are.

BTW, I would be perfectly fine with you stopping this interaction.  I still owe you that technical answer but otherwise, I am getting tired of dealing with another person's emotional outbursts.


oh, the hilarity