List of typical problems and shortcoming of "common" audio t Reply #25 – 2015-12-18 17:19:32 Quote from: Soap on 2015-12-18 16:40:06But you haven't identified any flaws in audio testing. You've identified flaws in people who you can't trust!Just like you couldn't trust patients and doctors alike before blind testing were introduced in medical testing.I just want you to focus on the following until it becomes clear to you (or ask for clarification, because this is central, and I hope we will agree):What you are saying and summing up by "you can't trust people" is correct.What you say right after that is we already have a solution to "you can't trust people".What I am saying is that "you can't trust people" is not the root problem, nor an interesting one: there are reasons that "you can't trust people".These reasons (for instance, they can look at a spectrogram or forge the test log, etc.) all lead to the same consequence: "you can't trust people".What YOU tell me is that what I am doing is useless because it all comes down to "you can't trust people".What I am telling you is that acting directly on the reasons can lead to new audio tests, with new/other limitations and trade-offs.But first we have to identify the main problems (like "you can't trust people"), their causes (they can look at a spectrogram or forge the test log, etc.) and the consequences (uncompelling evidence).If I apply that to the medical field...The orginal problem was exactly the same: they couldn't trust the doctors (not that they would say if the drug was true or fake, but some would ask "are you sure there is absolutely no improvement since last week?" to a person with a 'real pill" while they would not insist for the persons with the fake ones).In your current way of thinking, tell me if I am wrong, if I were to say "ok, so we don't trust doctors because 1-they tend to use different questions in each case 2- they use different overall tones 3- they tend to insist if the result does not go expected way (are you sure you really feel better? vs are you sure there is absolutely no improvement whatsoever?) 4- some blatantly cheat, etc.", you would take make list, find counter arguments and finally conclude, "well, in the end, it all comes down to 'we can't trust the doctors'".However, there is some value to find the cause the this lack of trust: by doing so, you can improve, not only locally (by putting a nurse behind each doctor to check for flaws and avoid cheating for instance) but globally ("interestingly enough, these 4 cause-problem disappear if the doctors themselves do not know what they are giving"). And naturally, if you remove the causes, the consequence vanishes as well.To get back to our case, having the cause (e.g. periodogram, fake log, etc.) give more insight and angles to strike than just saying that it all comes down to "we can't trust people".