One of these:
maybe if i remove some of the people who rated the lowest sample higher then cd it would be more interesting, and I think i have found something unusual....
You can't cherry pick raw data under any circumstances. It makes the test invalid regardless of your intentions, good or evil. If there was poor wording or a misunderstanding in the instructions, then you need to conduct a fundamentally new test, not discard raw data you "believe" to be compromised.
Disregarding all listeners who rated WAV as less than 5
Since you’ve just reminded me of something I wondered about earlier: how about disregarding all respondents whose data sets included each number only once? Or am I getting desperate here?
You shouldn't cherry pick raw data under any circumstances in a properly unbiased, double blind test. It makes the test suspect, regardless of the test conductor's intentions, good or evil. If there was poor wording or a misunderstanding in the instructions, then one needs to conduct a fundamentally new test, not discard raw data one "believes" to be compromised.
As far as I know, there is no universally agreed-upon treatment of outliers
As always, if you discover there is flaw it the test design, then you chuck ALL the data in the trash bin and re-design a new test.
basically just you unsupportable opinion
If there has been some kind of procedural error and it's not feasible to re-run the experiment, it's entirely legit to restrict your data down to the valid subset, if there is some easy way to do so. ...
"Cherry picking" is a type of confirmation bias, more accurately called a "fallacy of suppressed evidence" and may very well be unconscious in nature, despite its sinister sounding name. I wasn't, however, trying to speak poorly of anyone here or question their motives, but I seem to be alone here in thinking that claims of "pure and unbiased" motivation, which of course all scientists think applies to them , doesn't suddenly make cherry picking "acceptable". Everyone thinks their selection process is "sound, pure, and motivated only by the unbiased pursuit of truth".
Quote from: saratoga on 03 July, 2012, 05:05:09 PMbasically just you unsupportable opinionCareful, we've stepped below science into its superstructure: philosophy of science.
QuoteAs far as I know, there is no universally agreed-upon treatment of outliers You count them.