At a glance i think "markedly beyond" is exaggerated.

Quote

Besides being self-evident, it's also a very cheap get-out-of-jail answer in such a discussion.

"cheap get-out-of-jail answer" might be a suitable phrase for forums, but in science?

At a glance i think "markedly beyond" is exaggerated.

As a layman with no knowledge of statistics and little of scientific method, would I be far off in understanding this meta-analysis as amounting to something like...

have nothing to new to offer,  for some reasons or none, one might as well gather together and quote other work on what people call High-Resolution Audio. None of it conclusively shows any advantages, except, perhaps to certain rare beings with unusual hearing ability, but hey, lets go on talking about it.

If the aim was to keep people talking about it, it seems to have been effective.

Seriously folks, if we're trying to decide whether a high-res album sounds different from a CD 16/44 (of the same mastering of course), it should not need a meta-analysis. As a consumer, I can go on HDTracks this morning and see that a 24/192 version of Eric Clapton's recent album I Still Do costs US$27.98. And the CD on Amazon is US$10.90. It looks like both the CD and download are from the same DR11 master. The question for me in considering the purchase is not whether they may sound different, but rather does this difference justify a 250% markup!? In this context, does a 52.3% accuracy rate in a research setting sound like a valuable proposition to grab the high-resolution version?

You know guys, the fact that we're even going through the contortions of complex statistical analysis after >15 years since the release of SACD and DVD-A clearly indicates that those who claim to hear "obvious" differences are plainly wrong. When a meta-analysis is used in science to gather data far and wide to find and declare statistical significance of this kind of tiny magnitude, it just means that the "signal to noise" ratio is poor and that the magnitude of the effect is obviously academic. The author stated just as much: "In summary, these results imply that, though the effect is perhaps small and difficult to detect, the perceived fidelity of an audio recording and playback chain is affected by operating beyond conventional consumer oriented levels." Notice the careful wording... In no way does it imply that these "small" and "difficult to detect" differences are necessarily "better" as audiophiles always desire to promote. I like this wording and think Dr. Reiss did a fantastic job putting this together. By the way, these results are of no surprise as we've been talking about this for years!

See, the paper reached its goal and makes people accept now that some people have unusual abbilities.
Audiophiles have unusual abbilities and have many wordings for what they hear, others are ignorant.
The abbility is nowhere explained at a level i understand. It may only be these people catch some distortion in the audible band only present with high samplerates.

Archimago expressedmy feelings almost  exactly (bolding mine)

Quote

I like this wording and think Dr. Reiss did a fantastic job putting this together. By the way, these results are of no surprise as we've been talking about this for years!

Seriously folks, if we're trying to decide whether a high-res album sounds different from a CD 16/44 (of the same mastering of course), it should not need a meta-analysis. As a consumer, I can go on HDTracks this morning and see that a 24/192 version of Eric Clapton's recent album I Still Do costs US$27.98. And the CD on Amazon is US$10.90. It looks like both the CD and download are from the same DR11 master. The question for me in considering the purchase is not whether they may sound different, but rather does this difference justify a 250% markup!? In this context, does a 52.3% accuracy rate in a research setting sound like a valuable proposition to grab the high-resolution version?

I can imagine if the CD was done with best quality in mind it may be possible to sound even better as the HiBitrate download because of some watermark used for UMG related labels and online distribution.

I was not equating rare beings with audiophile. Audiophiles may be rarer than the makers of expensive kit and the publishers of over-priced music formats would like them to be, but they are not rare enough!

Audiophiles that read the Reiss paper will feel rare enough

<snip>
Well, this applies to an individual study. For a meta-analysis, I would expect the "independent variable" to be the same for each input study. If it isn't, I don't know anymore what the result actually says, nor whether the combining of the individual results makes sense. That may be my fault, and perhaps someone manages to enlighten me here.

<snip>
However, the conclusion would be wrong, because it would have been the result of the flaws in the tests. In each case the subjects' ability to distinguish the stimuli was because of secondary effects that compromised the test's internal validity.

So even though the individual tests reached the right conclusion, because the error wasn't strong enough to tip the balance, the meta analysis results in a wrong result.

That's why I directed the argument at you. Reiss is of couse entitled to recommend further research. I just would have wished he didn't oversell his own results.

That was my state of knowledge at the point in time when I wrote it.

Quote from: krabapple on 2016-07-10 05:18:04

Archimago expressedmy feelings almost  exactly (bolding mine)

It is of course the press release that will have the greatest impact with the public. A cursory search of the internet reveals how many media outlets have promptly picked this up. It doesn't look as if many of them had bothered to read the actual paper beyond the abstract, let alone put it under scrutiny. The almost universal message is that the paper supports the claims of audiophiles and the respective manufacturers. This applies even to the media that aren't associated with the audiophile sector.

Given what Archimago quite rightly pointed out, it should come as no surprise that the audiophile scene craves desperately for scientific confirmation of their daydreams. The fact that this is brewing for 20 years with no clear result, is an embarassment of the highest order when you try to build a mass market on it. No wonder there are attempts like those whe are discussing here.

Quote from: pelmazo on 2016-07-09 16:47:21

<snip>
Well, this applies to an individual study. For a meta-analysis, I would expect the "independent variable" to be the same for each input study. If it isn't, I don't know anymore what the result actually says, nor whether the combining of the individual results makes sense. That may be my fault, and perhaps someone manages to enlighten me here.

As stated before, in some cases combining results in the way Reiss did, is justified. If the CD-format is really transparent,

Please reread my posts, as i did address that point already.

What was meant by "combining" is, that combining studies related to "more bits" or "higher sampling rate" or "more bits and higher sampling rate" is justified under the constraints already explained.

I´d say, a bit less drama and provision of more arguments instead could help.

I´d say, a bit less drama and provision of more arguments instead could help.

I totally agree.....

Did you really expect anything more from this intellectually dishonest placebophile apologist?

If the CD-format is really transparent..

As stated before, in some cases combining results in the way Reiss did, is justified. If the CD-format is really transparent, it doesn´t matter if there are more bits or higher sampling frequency or both, because more than "transparent" is not possible. If there is nevertheless an effect then it is an useful result.

Why should you expect that a meta-analysis has to deliver an answer to any further question?

As said before, it is quite common that authors of meta-analysis (or systematic reviews) do strongly recommend further research.

I am a bit puzzled at this point; if you didn´t reanalyzed the papers used and didn´t do the statistics to see which effect inclusion or exclusion of various papers might have, how do you know that he "oversells his results" ?

That why i wrote you "missed" . You could and should have known better.

For the record, I have done dozens of DBTs and evaluated my own ability to differentiate between "cd-format" and "hi-res". It was null and it has been null for all the persons that I have seen testing it in the same way.