Since some people on this forum hold a certain reverence for ABX testing, I decided to play around with it. I know that doing this isn't going to change anyone's mind, and I will be just be accused for cheating if I did "pass" ABX testing anyway so it's literally a lose-lose situation. But still, I'm interested, the tools are easily available, and following some famous successes at ABX-ing high-resolution audio, I went and downloaded the ABX music set from AVS -- I didn't read a single post on that thread, just scrolled down to the download link and clicked it. I didn't cheat in any way.

Going into this, I knew it would be tough -- I tried A/B-ing a 24/96 vs 16/44.1 version of one track I had in my library and I simply could not tell the difference. A/B-ing the down-sampled DSD-256 file I discussed early in another thread was much more significant. Doing it blind with songs I have never heard before, I wouldn't be surprised if I couldn't hear anything at all. Now, this was a first run, just to kill an hour, so I didn't put the track through an extensive critical listening regime, I just stopped at a section I thought the difference was pronounced (symbol taps starting at 01:20) and took the test from there. Let me just say, the difference is SMALL. Very slight. It wasn't until half-way through the test that I thought I had finally nailed down what to listen for -- but at that point, audio memory was smearing things, so towards the end I was reduced to just going with my gut reaction on what I felt was that tad more open on the sound signature of the cymbals. I made these judgement on a 15 second snippet of the track, trying to determine them based solely on the symbol hits.

Quote

foo_abx 2.0.2 report
foobar2000 v1.3.9
2016-03-14 19:31:10

File A: Just_My_Imagination_A2.wav
SHA1: 2a913086b5e4c2fa052e643a2ad11c18ea598cff
File B: Just_My_Imagination_B2.wav
SHA1: 654dcecc9137b29f980d8d28fd63b5470b4695dd

Output:
ASIO : XMOS USB Audio 2.0 ST 2023
Crossfading: NO

19:31:10 : Test started.
19:33:28 : 00/01
19:36:19 : 00/02
19:38:05 : 00/03
19:39:33 : 00/04
19:44:35 : 00/05
19:45:22 : 01/06
19:47:09 : 02/07
19:49:28 : 03/08
19:52:20 : 04/09
19:54:22 : 05/10
19:54:22 : Test finished.

 ----------
Total: 5/10

It wasn't until half-way through the test that I thought I had finally nailed down what to listen for

Beginning to smell like a troll.

However, those that listen to these amps can tell a tremendous difference that seem to have little to do with noise performance. Why? No one really knows

Since some people on this forum hold a certain reverence for ABX testing, I decided to play around with it.

I know that doing this isn't going to change anyone's mind, and I will be just be accused for cheating if I did "pass" ABX testing anyway so it's literally a lose-lose situation.

It wasn't until half-way through the test that I thought I had finally nailed down what to listen for -- but at that point, audio memory was smearing things, so towards the end I was reduced to just going with my gut reaction on what I felt was that tad more open on the sound signature of the cymbals.

Total: 5/10

Rather, it's up to research efforts to account for differences between perception and measurements, with the understanding that measurements are very limited, and many unknown factors remain.

For example, you might believe that the difference between a $200 3-lbs class D amp from a $10,000 160-lbs class A amp comes down to SNR, IMD and THD ratings, and a few other minor measurements. However, those that listen to these amps can tell a tremendous difference that seem to have little to do with noise performance. Why? No one really knows -- not even the designers of the high end amp, who end up designing products through trail and error with many different components until they find a combination that sounds good to them. The job of research efforts is to try and identify and measure the phenomena at play.

Oh, and passing off amateur hour stunts at convention meetings (visual bias) as "research" shouldn't count.