Perhaps the lack of killer samples for jitter reflects the lack of HDMI jitter being audible in practice for human listeners.  (I seem to recall Ethan Winer suggesting something to the effect that jitter should not normally be noticeable with modern equipment.)

If there is no source control for segment selection and looping, I don't think adding user control gets us to where we want to be.

Yes, it is better than timed switching.  But one can't compare the past to the future as was done with the ABX comparator itself.

Quote

Much of an ABX test, however, comes in the material selection, listener training, listener comfort, and presence of listener feedback.

A failing unfortunately for just about every test cited on audio forums as proof of this and that.  The outcomes may still be right but the protocol certainly not.

The name ABX was among other things a play on my initials. "ABK's ABX test". ;-)

I have never heard an audio file that incorporates an intentionally large amount of jitter, introduced for demonstration purposes, to hear what really excessive jitter actually sounds like.  (Perhaps someone here at HA might be kind enough to provide a link to such a file.)

Quote from: MLXXX on 2014-12-08 17:04:26

I have never heard an audio file that incorporates an intentionally large amount of jitter, introduced for demonstration purposes, to hear what really excessive jitter actually sounds like.  (Perhaps someone here at HA might be kind enough to provide a link to such a file.)

I now see that ABK in June this year furnished files for this forum demonstrating a range of levels of jitter severity: Jitter Listening test files.

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:39:16

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz Severe Jitter 0.05.flac

16:39:16 : Test started.
16:39:52 : 01/01 50.0%
16:40:26 : 02/02 25.0%
16:40:38 : 03/03 12.5%
16:40:48 : 04/04 6.3%
16:40:58 : 05/05 3.1%
16:41:07 : 06/06 1.6%
16:41:24 : 07/07 0.8%
16:41:32 : 08/08 0.4%
16:41:51 : 09/09 0.2%
16:42:04 : 10/10 0.1%
16:42:12 : 11/11 0.0%
16:42:21 : 12/12 0.0%
16:42:43 : Test finished.

----------
Total: 12/12 (0.0%)

=============
foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:33:19

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz max jitter 0.1.flac
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav

16:33:19 : Test started.
16:34:25 : 01/01 50.0%
16:34:38 : 02/02 25.0%
16:34:58 : 03/03 12.5%
16:35:16 : 04/04 6.3%
16:35:26 : 05/05 3.1%
16:35:41 : 06/06 1.6%
16:35:54 : 07/07 0.8%
16:36:34 : 08/08 0.4%
16:36:44 : 09/09 0.2%
16:36:54 : 10/10 0.1%
16:37:03 : 11/11 0.0%
16:37:13 : 12/12 0.0%
16:38:05 : Test finished.

----------
Total: 12/12 (0.0%)

============

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:44:39

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz jitter strong level .025.flac

16:44:39 : Test started.
16:45:05 : 01/01 50.0%
16:45:15 : 02/02 25.0%
16:45:28 : 03/03 12.5%
16:45:36 : 04/04 6.3%
16:45:54 : 05/05 3.1%
16:46:17 : 06/06 1.6%
16:46:29 : 07/07 0.8%
16:46:45 : 08/08 0.4%
16:46:55 : 09/09 0.2%
16:47:05 : 10/10 0.1%
16:47:19 : 11/11 0.0%
16:47:33 : 12/12 0.0%
16:47:36 : Test finished.

----------
Total: 12/12 (0.0%)

=====

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 19:04:40

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz noticable jitter 0.0125.flac

19:04:40 : Test started.
19:05:27 : 01/01 50.0%
19:05:54 : 02/02 25.0%
19:06:19 : 03/03 12.5%
19:06:35 : 04/04 6.3%
19:06:57 : 05/05 3.1%
19:07:16 : 06/06 1.6%
19:07:43 : 07/07 0.8%
19:08:15 : 08/08 0.4%
19:08:37 : 09/09 0.2%
19:09:05 : 10/10 0.1%
19:09:30 : 11/11 0.0%
19:10:05 : 12/12 0.0%
19:10:09 : Test finished.

----------
Total: 12/12 (0.0%)

That point aside, this is not a useful test of "jitter."  Arny has picked very low frequency of 30 Hz for the jitter which means masking is quite strong.

Sources of jitter can and do have wide ranging spectrum not represented by this one example.

Would be good to see if anyone here will attempt to run Arny's test and report on it.  I noticed no results reported in the thread he post here either.

Anyone thinking there is no such thing as critical listeners, should give the above a listen and report back.  I think they will find that they are mistaken in that regard .

Would be good to see if anyone here will attempt to run Arny's test and report on it.  I noticed no results reported in the thread he post here either.

Quote from: MLXXX on 2014-12-09 04:32:37

Quote from: MLXXX on 2014-12-08 17:04:26

I have never heard an audio file that incorporates an intentionally large amount of jitter, introduced for demonstration purposes, to hear what really excessive jitter actually sounds like.  (Perhaps someone here at HA might be kind enough to provide a link to such a file.)

I now see that ABK in June this year furnished files for this forum demonstrating a range of levels of jitter severity: Jitter Listening test files.

Arny post those on AVS Four a few months ago.  Here you can see me passing them .  http://www.avsforum.com/forum/91-audio-the...ml#post25081490

Quote from: Amir on AVS link=msg=0 date=

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:39:16

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz Severe Jitter 0.05.flac

16:39:16 : Test started.
16:39:52 : 01/01 50.0%
16:40:26 : 02/02 25.0%
16:40:38 : 03/03 12.5%
16:40:48 : 04/04 6.3%
16:40:58 : 05/05 3.1%
16:41:07 : 06/06 1.6%
16:41:24 : 07/07 0.8%
16:41:32 : 08/08 0.4%
16:41:51 : 09/09 0.2%
16:42:04 : 10/10 0.1%
16:42:12 : 11/11 0.0%
16:42:21 : 12/12 0.0%
16:42:43 : Test finished.

----------
Total: 12/12 (0.0%)

=============
foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:33:19

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz max jitter 0.1.flac
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav

16:33:19 : Test started.
16:34:25 : 01/01 50.0%
16:34:38 : 02/02 25.0%
16:34:58 : 03/03 12.5%
16:35:16 : 04/04 6.3%
16:35:26 : 05/05 3.1%
16:35:41 : 06/06 1.6%
16:35:54 : 07/07 0.8%
16:36:34 : 08/08 0.4%
16:36:44 : 09/09 0.2%
16:36:54 : 10/10 0.1%
16:37:03 : 11/11 0.0%
16:37:13 : 12/12 0.0%
16:38:05 : Test finished.

----------
Total: 12/12 (0.0%)

============

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:44:39

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz jitter strong level .025.flac

16:44:39 : Test started.
16:45:05 : 01/01 50.0%
16:45:15 : 02/02 25.0%
16:45:28 : 03/03 12.5%
16:45:36 : 04/04 6.3%
16:45:54 : 05/05 3.1%
16:46:17 : 06/06 1.6%
16:46:29 : 07/07 0.8%
16:46:45 : 08/08 0.4%
16:46:55 : 09/09 0.2%
16:47:05 : 10/10 0.1%
16:47:19 : 11/11 0.0%
16:47:33 : 12/12 0.0%
16:47:36 : Test finished.

----------
Total: 12/12 (0.0%)

=====

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 19:04:40

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz noticable jitter 0.0125.flac

19:04:40 : Test started.
19:05:27 : 01/01 50.0%
19:05:54 : 02/02 25.0%
19:06:19 : 03/03 12.5%
19:06:35 : 04/04 6.3%
19:06:57 : 05/05 3.1%
19:07:16 : 06/06 1.6%
19:07:43 : 07/07 0.8%
19:08:15 : 08/08 0.4%
19:08:37 : 09/09 0.2%
19:09:05 : 10/10 0.1%
19:09:30 : 11/11 0.0%
19:10:05 : 12/12 0.0%
19:10:09 : Test finished.

----------
Total: 12/12 (0.0%)

Only one other person attempted to run then and he only passed the easiest one although my memory is not very clear.  I could not get Arny to run and report his results or for that matter, anyone else.

That point aside, this is not a useful test of "jitter."  Arny has picked very low frequency of 30 Hz for the jitter which means masking is quite strong.  Sources of jitter can and do have wide ranging spectrum not represented by this one example.

Would be good to see if anyone here will attempt to run Arny's test and report on it.  I noticed no results reported in the thread he post here either.

Anyone thinking there is no such thing as critical listeners, should give the above a listen and report back.  I think they will find that they are mistaken in that regard .

I find foobar ABX plug-in woefully inadequate for finding small differences.

Here you can see me passing them . 

Quote from: Amir on AVS link=msg=0 date=

foo_abx 1.3.4 report
foobar2000 v1.3.2

Why is the no jitter file in your tests a wav while the others are flac? On Arnys download all is flac.

Maybe no jitter.wav was by accident a file with silence?

Can you repeat this with the new abx plugin?

Quote from: amirm on 2014-12-09 06:00:03

I find foobar ABX plug-in woefully inadequate for finding small differences.

Here you can see me passing them . 

Quote from: Amir on AVS link=msg=0 date=

foo_abx 1.3.4 report
foobar2000 v1.3.2

Well, which is it?
It seems ABX works just fine for showing audible differences, when on the agenda.

Quote from: Arny on AVS link=msg=0 date=

FOOBAR2000 has its Set Start and Set End feature to allow people to exploit this sort of thing. That's the standard implementation of this kind of feature and is IME about as good as it gets.

As good as it gets? I think it is one of the worst implementations. So that everyone can follow us, here is the sample picture of the UI:

First problem is that when you move the slider it does not update the time code. You have to hit the Start or End buttons to see it. This means a bunch of hunting with a slider to try to get it back to the same segment.

The slider cannot be stretched which means that it is very difficult to get it down to tenth of a second which is what you need when you want to isolate say, a single guitar pick.

When you hit the "set start" button nothing happens. It actually remembers that point but it shows no visual notification. It is only when you hit the end button that it shows where the start was. I want to see that marker so that for example I can give myself 1 second after that. Without seeing where the start position is, I wind up with a pain in the neck trial and error.

Then there is lack of looping. It plays from start to end of that segment but won't repeat. You want it to repeat so that you then switch inputs and hear their effects.

The other problem of switching tracks. There is no soft dissolve so you get a click as you switch between the input tracks.

Compounding all of this is that every time you start the tool it forgets it settings so all the pain repeats to adjust the slider.

And then there is the undiscoverable way of saving the results. You have to hit exit and then it gives you a chance to save the results. The file name however is blank. It should instead start with the file names. The ABX box often hides the tracks and when you are in the save dialog box, you cannot move it to see your track names.

All of this make it a lot harder to setup the right test. I gave up and just used the first segment I found. It was just too much work to try to locate others.

So if one wants to have less revealing results, you can say this is a good tool. Otherwise, it leaves a lot to be desired versus ones written by people who do this for a living.[/color]

Quote from: Arny on AVS link=msg=0 date=

If FOOBAR2000 is the worst ABX Comparator, then you should be able to quite easily provide us with a link to a similarly-priced but far better product.

Good morning Arny. I see you were up late posting these messages. Hopefully you didn't do it on my account .

Anyway, back to your post, you said and I quote: "FOOBAR2000 has its Set Start and Set End feature to allow people to exploit this sort of thing. That's the standard implementation of this kind of feature and is IME about as good as it gets. "

I responded by explaining how those features are poorly implemented and gave examples of why. My reference is proper tools designed in-house for this kind of work and spending years using them and realizing the benefits of the features that I talked about. And it was not just our tools at Microsoft that was this way. Same was true of the tools at other companies like Dolby that I had used in independent shootouts. That is my reference and not some other free tool you are asking me to produce.

All the failings of Foobar ABX would tilt the game in favor of finding no difference. So I get why we defend it and call it "as good as it gets." To someone who has not used any other tools with superior abilities, they would accept that and then wonder why they are not hearing the differences that are really there, but hard to isolate with Foobar ABX.

Now that you know the right feature set you can do your own search and find the right tool. Look at their UI and see if they have the features I was talking about. Here is a good example of a free tool that covers a lot of the bases that Foobar ABX does not: http://lacinato.com/cm/software/othersoft/abx.



Right away you see the Loop button as highlighted in the above sample screenshot. If you use the tool you will see that the sliders work without having to play anything. Better yet, there are two of them so you can position them closely to each other. In Foobar, the slider handle is thick and there is only one of it. This means that if you place the start and stop at say one second apart, you can't even tell that you have done that! They handle covers that short start-stop blue line.

Notice the ability to set gains as to equalize volumes if necessary.

The "offset" comes in handy for your track specifically. You have that deafening tone at the beginning which you need to skip over. And this lets you do that.

Is it perfect? No. It uses Java runtime and while it says it doesn't need a separate download, it asked me to do exactly that. The link at Oracle downloads version 1.7 and the tool wants 1.8. Good luck finding that version on Oracle site. But you can find it and once there the program runs without any installations. Simply double click on the .exe and it runs.

As much as I despise Java, in this case it does enable cross platform ability from Linux to Mac.

Another miss is not being able to enter the timecode directly instead of using the sliders.

And the best part is that I don't have to look at the silly mask of an alien in my task bar from Foobar. 

Anyway go ahead and use it and then let us know if Foobar ABX is still as good as it gets Arny[/color].

Quote from: ajinfla on 2014-12-09 13:19:16

Well, which is it?
It seems ABX works just fine for showing audible differences, when on the agenda.

The discussion was not around "ABX" but the specific instantiation of it in foobar plug-in as you quoted in my post. 

I find foobar ABX plug-in woefully inadequate for finding small differences.

Here you can see me passing them . 

Quote from: Amir on AVS link=msg=0 date=

foo_abx 1.3.4 report
foobar2000 v1.3.2

I don't buy this "it's so difficult - how can anyone be expected to manage this" argument. I assume the context is that we're taking people who think sighted testing is OK, and are trying to get them to blind test? So they're coming from sighted tests where there's no easy way to listen to the same segment twice, no easy way to synchronise sources, sometimes no easy way to switch quickly at all etc etc. Then you give them a double-blind test, solve all these problems, but it's suddenly too hard to hear a difference? Come on. We both know why it's too hard to hear the difference: no one is telling them what to expect any more. Boo Hoo.

In these tests, I tried to increase the number of iterations in order to decrease the margin of error.

PC -> Optical S/PDIF -> Pioneer Elite SC-55 -> Sony MDR-1R

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/15 08:40:55

File A: E:\AVS\Foobar ABX\Jangling Keys\keys jangling band resolution limited 4416 2496.wav
File B: E:\AVS\Foobar ABX\Jangling Keys\keys jangling full band 2496.wav

08:40:55 : Test started.
08:41:50 : 01/01 50.0%
08:42:08 : 01/02 75.0%
[...]
08:47:00 : 13/29 77.1%
08:47:07 : 13/30 81.9%
08:47:13 : Test finished.

----------
Total: 13/30 (81.9%)


and...

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/07/15 09:14:40

File A: E:\AVS\Foobar ABX\Jangling Keys\keys jangling band resolution limited 3216 2496.wav
File B: E:\AVS\Foobar ABX\Jangling Keys\keys jangling full band 2496.wav

09:14:40 : Test started.
09:15:33 : 00/01 100.0%
09:15:45 : 00/02 100.0%
09:16:20 : 01/03 87.5%
[...]
09:22:30 : 11/29 93.2%
09:22:45 : 11/30 95.1%
09:22:48 : Test finished.

----------
Total: 11/30 (95.1%)

Mark Henninger

Laptop? Practice?
Well, I decided to give my laptop a try since Amir did so well using his. Lo and behold, I had little difficulty with the 16/32 key jangling test. Not quite perfect, but I suspect a bit more practice would get me up to perfect.

My laptop is a Sony Vaio PCG-41412L with the HD upgraded to a SSD. All audio enhancements are off. I used a pair of Sony MDR-1R headphones.

The results speak for themselves; I found a critical segment that revealed an audible difference. I've had some practice, which helped—just as Amir suggested. Now, I can pass an ABX test I previously failed. I'll tackle the 16/44 test next. Oh, and it was a piece of cake to pick out the differences in the 16/16 and 22/16 tests.

foo_abx 1.3.4 report
foobar2000 v1.3.3
2014/07/19 11:26:49

File A: C:\Users\mark_000\Downloads\keys jangling band resolution limited 3216 2496.wav
File B: C:\Users\mark_000\Downloads\keys jangling full band 2496.wav

11:26:49 : Test started.
11:27:29 : 00/01 100.0%
11:28:58 : 00/02 100.0%
11:29:46 : 00/03 100.0%
11:29:59 : 01/04 93.8%
11:30:06 : 01/05 96.9%
11:30:16 : 02/06 89.1%
11:30:26 : 03/07 77.3%
11:30:34 : 04/08 63.7%
11:30:45 : 05/09 50.0%
11:31:00 : 06/10 37.7%
11:31:10 : 07/11 27.4%
11:31:29 : 08/12 19.4%
11:31:41 : 09/13 13.3%
11:32:05 : 10/14 9.0%
11:32:20 : 10/15 15.1%
11:32:30 : 11/16 10.5%
11:32:41 : 12/17 7.2%
11:32:52 : 13/18 4.8%
11:33:07 : 13/19 8.4%
11:33:16 : 14/20 5.8%
11:33:28 : 15/21 3.9%
11:33:40 : 16/22 2.6%
11:33:58 : 17/23 1.7%
11:34:12 : 18/24 1.1%
11:34:25 : Test finished.

----------
Total: 18/24 (1.1%)

Mark Henninger

Quote from: Wombat on 2014-12-09 11:28:17

Can you repeat this with the new abx plugin?

Won't do.  But you can try and see if you can pass them as MLXXX just did.

Do you know what the group effort was composed of?

It really hasn't been discussed, but for example it included but was far from being limited to writing Dave Clark's AES convention paper and suquel JAES article. Where did I take credit for doing any of that?

I know that you are overwhelmed by envy, have never done anything of consequence in your profession, and have already tried and convicted me in your mind of being a thief of other people's intellectual property.

Quote from: Arnold B. Krueger on 2015-06-17 15:59:34

Do you know what the group effort was composed of?

Not exactly, that’s why I asked. But uh, see below.

Quote from: Arnold B. Krueger on 2015-06-17 15:59:34

It really hasn't been discussed, but for example it included but was far from being limited to writing Dave Clark's AES convention paper and sequel JAES article. Where did I take credit for doing any of that?

In the author list, of course! Are you one of the author’s? No? Why not, if you contributed so much?

Quote from: Arnold B. Krueger on 2015-06-17 15:59:34

I know that you are overwhelmed by envy, have never done anything of consequence in your profession, and have already tried and convicted me in your mind of being a thief of other people's intellectual property.

Your wild shot in the dark is too far off mark to even comment. But the fact that I asked “what credit do you give to others?” and “can you tell me how to find a source for the exact history”, and your response is all you, you, you, shows clearly how you blow your own horn.

So Arnold, you are the inventor of ABX?

Quote from: Rich B on 2015-06-17 22:16:09

So Arnold, you are the inventor of ABX?

Thanks for the question, as I can now answer it in a better state of being informed about the prior (ca. 1950) non-interactive ABX test.

I am the sole constructor of the first Interactive ABX Comparator, and the person who did the first Interactive ABX test.

That test compared the amplifier section of a Heath  AR1500 stereo receiver to a Dyna ST-400 power amp while driving Ohm F speakers

It came out random guessing.

I think I found you on the web, and I was then correct - you have nothing that compares with the entire ABX project in terms of scope and impact on the world of audio.  You can only wish!

Clark's part appears in publications under his name, Nousaine's part appears in publications under his name, but neither of them devised or built the part that I take credit for. 

Furthermore, it might be argued that there may have been some appropriation of other people's work but not by me.

Just another example of your ignorance. I am mentioned in both articles.

Quote from: Rich B on 2015-06-17 22:16:09

So Arnold, you are the inventor of ABX?

Thanks for the question, as I can now answer it in a better state of being informed about the prior (ca. 1950) non-interactive ABX test.

However the ASA was publishing a great many papers based on the JASA version of ABX long before the AES even mentioned the AES version of ABX.  People who read journals and don't just drop their names know such things.  They also know the nature of the differences between the two. One of the items in the committee review of Clark's AES ABX paper included clarifying these things because enough people in the AES knew about the ASA version of ABX whcih came much earlier.  So did we.

The entire ABX Project had many participants and extended over several decades, but by then the experimental design was fully settled and every machine that implemented it was like Foobar2000/ABX - yet another re-implementation of the original ABX Comparator concept that I made happen by myself, only assisted by the people who helped make it fail until it was ready for prime time. There was no "Why don't you try this?" As soon as we had failed I knew exactly what to do next and it worked as far as it went, until the next innovation was needed and I supplied that, too.

I'm not asking if you did the first test. I'm asking if you are the inventor of ABX. Not the switchbox.  Not the test. 

ABX.

Are you the inventor of ABX?