Jitter Listening test files

Topic: Jitter Listening test files (Read 45321 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

Jitter Listening test files

2014-06-20 01:02:59

I prepared some files for people who want to do ABX tests related to jitter.

The file archive is here:

https://www.dropbox.com/sh/b35feharwc7doty/...t9KwuBTPbBa1JZa

Among other things it contains the following files:

no jitter.flac
30 Hz max jitter 0.1.flac
30 Hz Severe Jitter 0.05.flac
30 Hz jitter strong level .025.flac
30 Hz noticable jitter 0.0125.flac
30 Hz jitter marginal level .00625.flac
30 Hz threshold jitter 0.00312.flac

The intent that these files be ABXed using the FOOBAR2000 ABX comparator starting with the max jitter file. The "max jitter" file is painfully easy to ace but is a good warm up. All but the last two files can be aced but at some point it will start taking a little work.

Allow about 7 minutes for listening to each of the first 4 files, about 12 minutes for the fifth one, and

for the remaining 2.

The files start out with a 1 KHz sine wave, with the same amount of jitter as the rest of the file. One can use FOOBAR2000 to select it or ignore it. You can FFT it to see how the jitter was applied.

Jitter Listening test files

Reply #1 – 2014-06-20 15:29:14

Thank you.

At this time, I'm more interested in the question what does jitter sound like anyway than in finding out any personal detection thresholds. Maybe that can come later

The world is full of people who talk about it and how much they have to spend to avoid it --- but they never play me any!

Jitter Listening test files

Reply #2 – 2014-06-20 20:20:55

Quote from: Thad E Ginathom on 2014-06-20 15:29:14

Thank you.

At this time, I'm more interested in the question what does jitter sound like anyway than in finding out any personal detection thresholds. Maybe that can come later

The world is full of people who talk about it and how much they have to spend to avoid it --- but they never play me any!

Then compare these two files:

no jitter.flac

30 Hz max jitter 0.1.flac

The audible difference is hard to miss, and is due to a real world kind of jitter that has been commonly found in AVRs that process HDMI A/V signals.

Jitter Listening test files

Reply #3 – 2014-06-20 20:43:36

Thanks for these, Arnold.

What do the decimal numbers at the end of the file names correspond to? I realize it's the relative amount of jitter, but can they be correlated with a measurement in picoseconds?

Jitter Listening test files

Reply #4 – 2014-06-20 21:52:49

Quote from: mixminus1 on 2014-06-20 20:43:36

Thanks for these, Arnold.

What do the decimal numbers at the end of the file names correspond to? I realize it's the relative amount of jitter, but can they be correlated with a measurement in picoseconds?

I don't like to reveal too much about the files because that may bias listeners.

What I can reveal up front is that:
1. The jitter in the "Max jitter" file will be audible to all,
2. The added jitter drops by about 50% between the files, and
3. The last two or three files will be difficult or impossible to distinguish from the "no jitter" file.

Each file starts out with a 1 KHz test tone (that you can edit out of the listening test using FOOBAR's Set Start and Set End features) that a knowledgeable person can analyze with a FFT to determine the exact kind and amount of added jitter. All files were processed consistently across that file, so there is no "trust me" in this story.

It turns out that the current convention of measuring jitter in picoseconds is pretty meaningless because it is independent of the frequency of the test signal, while the audibility of jitter measured in picoseconds is dependent on the frequency of the test signal.

But I know what it is for the files, anyway. The numbers are relevant to how the jitter was added to the files.

Jitter Listening test files

Reply #5 – 2014-06-21 02:24:39

Fair enough - thanks for the info!

Jitter Listening test files

Reply #6 – 2014-12-09 11:18:29

ABK,
thanks for these jitter files. Here are my listening results, and comments:-

Maximum jitter file
I was able to ABX this very quickly against the nil jitter file:

foo_abx 1.3.4 report
foobar2000 v1.1.13
2014/12/09 18:45:02

File A: \\readyshare\usb_storage\jitter\30 Hz max jitter 0.1.flac
File B: \\readyshare\usb_storage\jitter\no jitter.flac

18:45:02 : Test started.
18:45:51 : 01/01 50.0%
18:46:03 : 02/02 25.0%
18:46:12 : 03/03 12.5%
18:46:22 : 04/04 6.3%
18:46:32 : 05/05 3.1%
18:46:44 : 06/06 1.6%
18:46:49 : Test finished.

----------
Total: 6/6 (1.6%)

I heard a pervasive rasping quality to the 1kHz test tone, the guitar, and the vocal.

Threshold jitter level file
The jitter in the threshold jitter level file was too low for me to perceive it reliably, if at all.

Marginal jitter level file
The jitter in the marginal level jitter file was perceptible. It took me much longer to do this test, as I needed breaks to refresh my hearing:

foo_abx 1.3.4 report
foobar2000 v1.1.13
2014/12/09 19:04:35

File A: \\readyshare\usb_storage\jitter\30 Hz jitter marginal level .00625.flac
File B: \\readyshare\usb_storage\jitter\no jitter.flac

19:04:35 : Test started.
19:07:03 : 01/01 50.0%
19:07:33 : 02/02 25.0%
19:15:59 : 03/03 12.5%
19:42:16 : 04/04 6.3%
20:03:51 : 05/05 3.1%
20:04:11 : Test finished.

----------
Total: 5/5 (3.1%)

I didn't find the 1kHz test tone much help in distinguishing the files at this relatively low level of jitter. I mostly used the last few seconds of the recording where the singer can be heard producing a somewhat unstable vibrato. She synchronises her pulsating vibrato with the beats of the music. For my ears, the vibrato sounds slightly more harmonious and "direct" in the nil jitter version. Also the guitar sounds slightly cleaner and less "woolly" in the nil jitter version. However there was very little in it for my ears.

Noticeable jitter level file
I found the jitter in this file a little distracting.

Strong jitter level file
I found the jitter level quite unpleasant. I would categorise this as an unacceptable level of jitter, even for casual listening.

Discussion
Have I heard this type of "jitter" sound before? No, not that I can recall. Not with any digital audio equipment over the years.

With the 1kHz test tone, the higher amplitude jitter reminded me of a very fast vibrato of a musical instrument.

What equipment did I use? For preliminary testing I used a Xonar sound card and headphones. I was able to establish that the threshold level jitter file had too subtle an effect for me to be able to succeed with an ABX test. I then used a different pc* with onboard HDMI driving a low cost AVR.** (I was conscious of the irony of using an HDMI connection, in view of current discussion in the History and Accreditation of ABX Testing/Invention? thread starting at post #47.) Using the AVR fed from the pc, I could hear the effect of the artificially created jitter in the test files, either with headphones or loudspeakers.

It will be interesting if ABK is able to disclose for us what level of jitter was present in his "threshold jitter level file", and how that broadly compares with the performance of real world HDMI connected devices.

I note as a general comment that there is always the possibility of reducing jitter anyway in a playback device by use of a small buffer and a steady clock with a relatively slow time constant for fine adjustments to its clock rate, if there is any doubt about the short term timing stability of a stream of incoming digital audio. What extent of buffering is commonly in use I have no idea.

____________

* This pc has the old version of the foobar ABX plug-in. For tests demanding very careful listening, I appreciate being able to see my progressive results. The replacement foobar ABX plug-in under development, when I last checked, conceals progressive results when in formal test mode. I have lodged a request that there be an option to display progressive results.
** The AVR used was a Pioneer VSX-820.

Jitter Listening test files

Reply #7 – 2014-12-09 13:37:57

Quote from: MLXXX on 2014-12-09 11:18:29

It will be interesting if ABK is able to disclose for us what level of jitter was present in his "threshold jitter level file", and how that broadly compares with the performance of real world HDMI connected devices.

Thank you for your work and comments.

The presence of the 1 KHz tone makes the files self-disclosing. Use your favorite FFT tool to look at the sidebands, and there you are. In the case of this file the first set of sidebands are at +/- 30 Hz with amplitudes about 66 dB below the carrier.

More readings on the topic:

The scientific literature (example Zwicker and Fastl) suggests that FM distortion is heard based on three different perceptual models depending on jitter frequency. At very low jitter frequencies, the ear simply tracks the changes. LP rotation-based jitter is usually perceived this way. Above a few Hz jitter is heard as roughness which my 30 Hz jitter samples demonstrate quite clearly. Above a few 100 Hz masking starts coming into play.

Reference Zwicker and Fastl Psychoacoustics Facts and Models (Third Edition, 2007):

Zwicker and Fastel's Psychoacoustics Facts and Models PDF

Chapter 10, page 258 paragraph 10.1

Here is the information that inspired my choice of 30 Hz as my first test frequency

Straight from Amir's own tests!

With the sideband frequencies annotated:

Please notice that the largest set of sidebands are at +/- 30 Hz, indicating a jitter frequency of 30 Hz.

The complaints from certain quarters over this choice of test frequencies seems to be in agreement with the old saw: "No good deed goes unpunished". I used the guy's own selected data as a guide and see what I get! ;-)

Here is an example of a digital audio produce with relatively high amounts of jitter:

As you can see the sidebands are about 80 dB down and are at +/- 3.0 Khz.

or here:

a fairly modern Onkyo TXNR 1009 AVR's HDMI jitter with all sidebands > 100 dB down (plenty fine).

http://2.bp.blogspot.com/-d5banBFYrqI/UmSY...HDMI+Jitter.png

Or apparently the same AVR under what may be slightly different test conditions with the first sidebands +/- 120 Hz at about 90 dB down (still OK).

http://amirviews.smugmug.com/photos/i-tn5r...-tn5rJfr-X3.png

Quote

I note as a general comment that there is always the possibility of reducing jitter anyway in a playback device by use of a small buffer and a steady clock with a relatively slow time constant for fine adjustments to its clock rate, if there is any doubt about the short term timing stability of a stream of incoming digital audio. What extent of buffering is commonly in use I have no idea.

Agreed. the actual jitter in any audio reconstructed from the digital domain is reducible to any desired level. My impression is that most modern designers are guided by influences similar to this paper "‘Theoretical and audible effects of jitter on digital audio quality,’’ Preprint of the 105th AES Convention, #4826 (1998).E. Benjamin and B. Gannon.

Engineers can reduce it arbitrarily but since arbitrarily large amounts of reduction tend to be arbitrarily expensive, they just put it well below what they expect and/or find it to be audible at.

Jitter Listening test files

Reply #8 – 2014-12-09 13:57:35

The files contain tags, including a title. The title for the "threshold" file says "max" I wasn't ABX'ing, just curious, and was most confused that I couldn't hear anything at all in the max file. Because I was listening to the threshold case

Jitter Listening test files

Reply #9 – 2014-12-09 14:26:52

Quote from: MLXXX on 2014-12-09 11:18:29

I note as a general comment that there is always the possibility of reducing jitter anyway in a playback device by use of a small buffer and a steady clock with a relatively slow time constant for fine adjustments to its clock rate, if there is any doubt about the short term timing stability of a stream of incoming digital audio. What extent of buffering is commonly in use I have no idea.

Buffering gets used in every device out there. The problem with slow adjustment is the tracking range is very wide. A receiver needs to lock onto sampling rates from 32 Khz to 192 Khz. If it goes too slow, when you switch sources, you will have to wait many seconds for it to find and lock onto the incoming sample rate. Customers will not accept input switching delays of more than 1 or 2 seconds whereas they are oblivious to jitter as a problem so fast switching wins.

Here is a high-resolution spectrum of jitter in a Pioneer AVR as measured by Paull Miller of HiFi news:

Everything but the center tone are distortion/jitter sidebands. We see tons of components there, not just one "30 Hz" frequency Arny has picked. 30 Hz is representative of what you hear in analog systems. What governs its audibility at the very high levels is temporal masking. The peak is 4 Khz so 30 Hz is not quite the peak but nevertheless, is not symptomatic of modern manifestation of jitter which can have arbitrary components.

There are solutions to this but it requires design skills, and a few dollars worth of parts:

At the other extreme we have these horrid measurements that I made of this $1000 premium AVR:

As the graph indicates, these are the exact same measurements of the same device just a few seconds apart. The AVR randomly induces tons more jitter into the DAC clock resulting in much increased low frequency random jitter and increase in correlated distortions.

This is just bad engineering taking advantage of customers not having any visibility into this performance aspect of device. Video magazines don't measure jitter. Audio magazines which do measure jitter don't care about AVRs.

Jitter Listening test files

Reply #10 – 2014-12-09 14:33:39

Quote from: lithopsian on 2014-12-09 13:57:35

The files contain tags, including a title. The title for the "threshold" file says "max" I wasn't ABX'ing, just curious, and was most confused that I couldn't hear anything at all in the max file. Because I was listening to the threshold case

I'll try to do better next set of files. I may get some time to correct the tags in this set.

Jitter Listening test files

Reply #11 – 2014-12-09 14:35:56

Quote from: amirm on 2014-12-09 14:26:52

Quote from: MLXXX on 2014-12-09 11:18:29
I note as a general comment that there is always the possibility of reducing jitter anyway in a playback device by use of a small buffer and a steady clock with a relatively slow time constant for fine adjustments to its clock rate, if there is any doubt about the short term timing stability of a stream of incoming digital audio. What extent of buffering is commonly in use I have no idea.

Buffering gets used in every device out there. The problem with slow adjustment is the tracking range is very wide. A receiver needs to lock onto sampling rates from 32 Khz to 192 Khz. If it goes too slow, when you switch sources, you will have to wait many seconds for it to find and lock onto the incoming sample rate. Customers will not accept input switching delays of more than 1 or 2 seconds whereas they are oblivious to jitter as a problem so fast switching wins.

Here is a high-resolution spectrum of jitter in a Pioneer AVR as measured by Paull Miller of HiFi news:

Everything but the center tone are distortion/jitter sidebands. We see tons of components there, not just one "30 Hz" frequency Arny has picked. 30 Hz is representative of what you hear in analog systems. What governs its audibility at the very high levels is temporal masking. The peak is 4 Khz so 30 Hz is not quite the peak but nevertheless, is not symptomatic of modern manifestation of jitter which can have arbitrary components.

There are solutions to this but it requires design skills, and a few dollars worth of parts:

At the other extreme we have these horrid measurements that I made of this $1000 premium AVR:

As the graph indicates, these are the exact same measurements of the same device just a few seconds apart. The AVR randomly induces tons more jitter into the DAC clock resulting in much increased low frequency random jitter and increase in correlated distortions.

This is just bad engineering taking advantage of customers not having any visibility into this performance aspect of device. Video magazines don't measure jitter. Audio magazines which do measure jitter don't care about AVRs.

Depends on whether you define bad engineering in terms of sound quality as heard by human beings listening to music and/or dialog or conformance with arbitrary numerical standards, no?

Jitter Listening test files

Reply #12 – 2014-12-09 14:40:58

Quote from: Arnold B. Krueger on 2014-12-09 14:33:39

Quote from: lithopsian on 2014-12-09 13:57:35
The files contain tags, including a title. The title for the "threshold" file says "max" I wasn't ABX'ing, just curious, and was most confused that I couldn't hear anything at all in the max file. Because I was listening to the threshold case

I'll try to do better next set of files. I may get some time to correct the tags in this set.

No problem. I think this is a great thread that sheds some light both on the jitter scaremongers and on those who claim it simply doesn't exist at detectable levels on modern audio systems.

Jitter Listening test files

Reply #13 – 2014-12-09 15:22:20

Posting these results here in addition to the parallel thread for others who only land here:
==============

http://www.avsforum.com/forum/91-audio-the...ml#post25081490

Quote from: Amir on AVS link=msg=0 date=

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:39:16

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz Severe Jitter 0.05.flac

16:39:16 : Test started.
16:39:52 : 01/01 50.0%
16:40:26 : 02/02 25.0%
16:40:38 : 03/03 12.5%
16:40:48 : 04/04 6.3%
16:40:58 : 05/05 3.1%
16:41:07 : 06/06 1.6%
16:41:24 : 07/07 0.8%
16:41:32 : 08/08 0.4%
16:41:51 : 09/09 0.2%
16:42:04 : 10/10 0.1%
16:42:12 : 11/11 0.0%
16:42:21 : 12/12 0.0%
16:42:43 : Test finished.

----------
Total: 12/12 (0.0%)

=============
foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:33:19

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz max jitter 0.1.flac
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav

16:33:19 : Test started.
16:34:25 : 01/01 50.0%
16:34:38 : 02/02 25.0%
16:34:58 : 03/03 12.5%
16:35:16 : 04/04 6.3%
16:35:26 : 05/05 3.1%
16:35:41 : 06/06 1.6%
16:35:54 : 07/07 0.8%
16:36:34 : 08/08 0.4%
16:36:44 : 09/09 0.2%
16:36:54 : 10/10 0.1%
16:37:03 : 11/11 0.0%
16:37:13 : 12/12 0.0%
16:38:05 : Test finished.

----------
Total: 12/12 (0.0%)

============

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 16:44:39

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz jitter strong level .025.flac

16:44:39 : Test started.
16:45:05 : 01/01 50.0%
16:45:15 : 02/02 25.0%
16:45:28 : 03/03 12.5%
16:45:36 : 04/04 6.3%
16:45:54 : 05/05 3.1%
16:46:17 : 06/06 1.6%
16:46:29 : 07/07 0.8%
16:46:45 : 08/08 0.4%
16:46:55 : 09/09 0.2%
16:47:05 : 10/10 0.1%
16:47:19 : 11/11 0.0%
16:47:33 : 12/12 0.0%
16:47:36 : Test finished.

----------
Total: 12/12 (0.0%)

=====

foo_abx 1.3.4 report
foobar2000 v1.3.2
2014/06/18 19:04:40

File A: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\no jitter.wav
File B: C:\Users\Amir\Music\Arny's 30 Hz Jitter File\Arny's new files\30 Hz noticable jitter 0.0125.flac

19:04:40 : Test started.
19:05:27 : 01/01 50.0%
19:05:54 : 02/02 25.0%
19:06:19 : 03/03 12.5%
19:06:35 : 04/04 6.3%
19:06:57 : 05/05 3.1%
19:07:16 : 06/06 1.6%
19:07:43 : 07/07 0.8%
19:08:15 : 08/08 0.4%
19:08:37 : 09/09 0.2%
19:09:05 : 10/10 0.1%
19:09:30 : 11/11 0.0%
19:10:05 : 12/12 0.0%
19:10:09 : Test finished.

----------
Total: 12/12 (0.0%)

Jitter Listening test files

Reply #14 – 2014-12-09 15:38:25

Quote from: lithopsian on 2014-12-09 14:40:58

I think this is a great thread that sheds some light both on the jitter scaremongers and on those who claim it simply doesn't exist at detectable levels on modern audio systems.

Pardon me, but how would anything in this thread ever relate to whether jitter is audibly detectable in modern audio systems?
Is it fair to assume that modern audio systems are used...for music?
Further, it has been shown repeatedly that online ABX files are highly suspect and susceptible to the incorrigible with agendas.
i.e., they can be gamed for $pecific purpose.
When I see where someone has audibly detected jitter listening to music on a modern audio system, blind and supervised, I'll pay some attention.

cheers,

AJ

Jitter Listening test files

Reply #15 – 2014-12-09 16:10:17

Quote from: ajinfla on 2014-12-09 15:38:25

Quote from: lithopsian on 2014-12-09 14:40:58
I think this is a great thread that sheds some light both on the jitter scaremongers and on those who claim it simply doesn't exist at detectable levels on modern audio systems.

Pardon me, but how would anything in this thread ever relate to whether jitter is audibly detectable in modern audio systems?
Is it fair to assume that modern audio systems are used...for music?
Further, it has been shown repeatedly that online ABX files are highly suspect and susceptible to the incorrigible with agendas.
i.e., they can be gamed for $pecific purpose.
When I see where someone has audibly detected jitter listening to music on a modern audio system, blind and supervised, I'll pay some attention.

cheers,

AJ

And that's just what I mean: lots of noise with zero information content. Simply your opinion that everything anyone says is crap. Well this thread actually contains something that isn't crap and still you just complain.

Jitter Listening test files

Reply #16 – 2014-12-09 19:14:50

Quote from: MLXXX on 2014-12-09 11:18:29

ABK,
thanks for these jitter files. Here are my listening results, and comments:-

Maximum jitter file
I was able to ABX this very quickly against the nil jitter file:

foo_abx 1.3.4 report
foobar2000 v1.1.13
2014/12/09 18:45:02

File A: \\readyshare\usb_storage\jitter\30 Hz max jitter 0.1.flac
File B: \\readyshare\usb_storage\jitter\no jitter.flac

18:45:02 : Test started.
18:45:51 : 01/01 50.0%
18:46:03 : 02/02 25.0%
18:46:12 : 03/03 12.5%
18:46:22 : 04/04 6.3%
18:46:32 : 05/05 3.1%
18:46:44 : 06/06 1.6%
18:46:49 : Test finished.

----------
Total: 6/6 (1.6%)

I heard a pervasive rasping quality to the 1kHz test tone, the guitar, and the vocal.

Threshold jitter level file
The jitter in the threshold jitter level file was too low for me to perceive it reliably, if at all.

Marginal jitter level file
The jitter in the marginal level jitter file was perceptible. It took me much longer to do this test, as I needed breaks to refresh my hearing:

foo_abx 1.3.4 report
foobar2000 v1.1.13
2014/12/09 19:04:35

File A: \\readyshare\usb_storage\jitter\30 Hz jitter marginal level .00625.flac
File B: \\readyshare\usb_storage\jitter\no jitter.flac

19:04:35 : Test started.
19:07:03 : 01/01 50.0%
19:07:33 : 02/02 25.0%
19:15:59 : 03/03 12.5%
19:42:16 : 04/04 6.3%
20:03:51 : 05/05 3.1%
20:04:11 : Test finished.

----------
Total: 5/5 (3.1%)

I didn't find the 1kHz test tone much help in distinguishing the files at this relatively low level of jitter. I mostly used the last few seconds of the recording where the singer can be heard producing a somewhat unstable vibrato. She synchronises her pulsating vibrato with the beats of the music. For my ears, the vibrato sounds slightly more harmonious and "direct" in the nil jitter version. Also the guitar sounds slightly cleaner and less "woolly" in the nil jitter version. However there was very little in it for my ears.

Are you setting your trial number *before* you start your tests?

IOW, in the first results shown did you actually decide beforehand, 'I will do 6 trials and stop' and for second results shown , "I will do 5 and stop'?

If not, in the future you should.

Certainly if you use progressive mode, where you can see your results as they happen, you should *not* stop at an arbitrary point....choose a number beforehand, and do the whole set.

Jitter Listening test files

Reply #17 – 2014-12-09 19:24:03

Quote from: lithopsian on 2014-12-09 14:40:58

No problem. I think this is a great thread that sheds some light both on the jitter scaremongers and on those who claim it simply doesn't exist at detectable levels on modern audio systems.

I don't know who those would be, since we can almost always detect *some level* of something if we look for it with sensitive enough instruments; I thought the argument was always about audibility.

And btw, if it turns out that one has to exhaust oneself, take coffee breaks, etc in order to hear a small amount of jitter 5/5 times in an ABX , I'm going to consider it a vanishingly small 'problem' for normal listening, and focus on things that matter instead.

That won't stop the scaremongers from doing what they do, of course. Scaring people about jitter sells hardware 'upgrades'.

Jitter Listening test files

Reply #18 – 2014-12-09 22:31:16

Quote from: krabapple on 2014-12-09 19:24:03

I don't know who those would be

The same folks who claim all amps, wires, DACs, etc, etc. "sound the same".
You know, them.

cheers,

AJ

Jitter Listening test files

Reply #19 – 2014-12-09 22:49:47

I ran the tests again just now and documented a specific procedure for passing the tests legitimately: http://www.hydrogenaud.io/forums/index.php...st&p=884128

Edit: Here is the new link to my results in the recycle bin: http://www.hydrogenaud.io/forums/index.php?showtopic=107718

Jitter Listening test files

Reply #20 – 2014-12-09 23:18:43

After complaining at amir i listened these samples the first time Peace of cake to abx down to 0.125.

foo_abx 2.0 beta 4 report
foobar2000 v1.3.4
2014-12-10 00:10:25

File A: 30 Hz noticable jitter 0.0125.flac
SHA1: 7349f9f6449777a2c33693251099a6ed58821382
File B: no jitter.flac
SHA1: 262cd6c4d4c73502a0142f867b00aae013fd13ce

Output:
DS : Primärer Soundtreiber

00:10:25 : Test started.
00:11:29 : 01/01
00:11:39 : 02/02
00:11:51 : 03/03
00:12:03 : 04/04
00:12:14 : 05/05
00:12:45 : 06/06
00:12:59 : 07/07
00:13:15 : 08/08
00:13:28 : 09/09
00:13:46 : 10/10
00:13:46 : Test finished.

----------
Total: 10/10
Probability that you were guessing: 0.1%

-- signature --
8c03ed033758c82830a239ea2a5f3f5ff12c910f

Jitter Listening test files

Reply #21 – 2014-12-10 00:07:50

Quote from: krabapple on 2014-12-09 19:14:50

Are you setting your trial number *before* you start your tests?

IOW, in the first results shown did you actually decide beforehand, 'I will do 6 trials and stop' and for second results shown , "I will do 5 and stop'?

If not, in the future you should.

Certainly if you use progressive mode, where you can see your results as they happen, you should *not* stop at an arbitrary point....choose a number beforehand, and do the whole set.

I think this argument is much stronger if the test subject rapidly undertakes a series of incomplete and unsuccessful trials. If the incomplete trials are excluded from the reported results, the report is misleading, by excluding the unsuccessful results.

The beta version of the new plug-in in the version I downloaded it a few weeks ago sets a minimum number of trials at 8. That is actually quite a high number for an acoustic difference only audible by a particular listener who needs their hearing to be fresh. The question arises: what is the test subject to do if a point is reached when they feel they can no longer hear a difference. Are they to answer recklessly without any belief in their answer? Or are they to abandon the exercise altogether at that point without any reportable result?

In a multiple choice exam for a university subject, there is an ethical question whether to answer questions by pure guesswork. The prevailing ethic (I believe) is to give an answer even to questions one has absolutely no idea about, the expectation being that others in the exam may do likewise, evening out the test results. I am not sure how that translates to ABXing audio. For myself, if I feel I cannot hear a difference, I would prefer either to wait until I felt I could hear a difference (if there seems a reasonable prospect my hearing will recover), or abandon the test altogether. It is surely a form of deception to provide an answer to an ABX test by pure guesswork without any inkling as to whether it is what one actually hears.

In this particular exercise, knowing my hearing's tendency to lose sensitivity when listening to the same material, I will disclose that I actually had in mind doing 6 trials for the marginal jitter level file. However when doing the trials I became aware it was becoming increasingly difficult to have confidence in my ability to distinguish. Given that 5% is often adopted in statistics as an acceptable level of confidence, I finished after 5 trials, with a probability of guessing of 3.1%. I felt the statistical significance was sufficient, and I didn't want to wait another 15 minutes or so for my hearing to regain sensitivity.

I also have an aversion to any wrong answer, or guessing. To my mind, I either hear a difference or I don't. To have provided a 6th answer before my hearing recovered (based on a 50% likelihood it would turn out to be "right" anyway) was simply not a path I wished to go down.

Quote from: krabapple on 2014-12-09 19:24:03

And btw, if it turns out that one has to exhaust oneself, take coffee breaks, etc in order to hear a small amount of jitter 5/5 times in an ABX , I'm going to consider it a vanishingly small 'problem' for normal listening, and focus on things that matter instead.

I strongly disagree here, krabapple. If one hears a problem when one's hearing is fresh, and provided the problem actually sounds "bad" to one's ears rather than just "subtly different", then to my mind it is a significant issue. It matters not that one doesn't hear it on another occasion when one's hearing is fatigued.

For example, by the time I got to the 5th trial I was beginning to lose my ability to distinguish the samples based on the vibrato of the last sung note.

So I tried instead listening carefully to the guitar (something I hadn't previously focussed on to any great degree). The jitter affected sample did sound somewhat mushy in comparison and so I was confidently able to answer. I would rank the woolliness of the guitar as a significant impairment of the sound, to be avoided if technologically feasible. Guitar accompaniment is not something I usually take much notice of, but if it were a flamenco guitar solo performance I would like to be able to hear the guitar clearly, without any introduced mushiness.

However isn't the measured jitter level of actual equipment using HDMI quite a bit less anyway? That is what I have taken away from this exercise. To the best of my knowledge I have never heard jitter effects outside of these test files Arny has provided.

Jitter Listening test files

Reply #22 – 2014-12-10 03:06:57

Quote from: amirm on 2014-12-09 14:26:52

Quote from: MLXXX on 2014-12-09 11:18:29
I note as a general comment that there is always the possibility of reducing jitter anyway in a playback device by use of a small buffer and a steady clock with a relatively slow time constant for fine adjustments to its clock rate, if there is any doubt about the short term timing stability of a stream of incoming digital audio. What extent of buffering is commonly in use I have no idea.

Buffering gets used in every device out there. The problem with slow adjustment is the tracking range is very wide. A receiver needs to lock onto sampling rates from 32 Khz to 192 Khz. If it goes too slow, when you switch sources, you will have to wait many seconds for it to find and lock onto the incoming sample rate. Customers will not accept input switching delays of more than 1 or 2 seconds whereas they are oblivious to jitter as a problem so fast switching wins.

If everyone is oblivious to the jitter then this would be an acceptable and appropriate outcome! However I assume you must be implying that some people can hear jitter in modern equipment.

I'm not privy to how phase-locked loops for audio streams are currently designed but I'd have thought it possible to have a coarse adjustment time constant to kick in if lock fails altogether such as when disconnecting and reconnecting an HDMI source, and a fine adjustment time constant to take over once approximate lock is achieved.

A mobile telephone connection would fall into a different category to playback of a Blu-ray disk. We have low expectations about audio quality for a mobile phone call and there may be packet loss depending on the instantaneous quality of the connection. An aggressive algorithm for phase lock may be appropriate in that application.

With Blu-ray, the source device should be clocking out the packets using a stable crystal controlled oscillator. The sink device could buffer using a slow time constant once lock was achieved. Similarly, a pc sound card should send HDMI packets with high precision to begin with. If the designer of an AVR wishes to tighten the timing even further by way of a buffer with a slow time constant, after approximate phase lock has been rapidly achieved with a fast time constant, that should be practical, I'd have thought.

Jitter Listening test files

Reply #23 – 2014-12-10 03:23:09

Quote from: MLXXX on 2014-12-10 03:06:57

I'm not privy to how phase lock loops audio streams are currently designed but I'd have thought it possible to have a coarse adjustment time constant if lock fails altogether such as when disconnecting and reconnecting an HDMI source, and a fine adjustment time constant once lock is achieved.

What you are describing is dual PLL solution which indeed accomplishes this goal. Alas, it is more expensive and complicated to design so you are going to see it in high-end gear, not mass market AVRs and such.

Here is a very revealing measurement I mad on a number of mass market AVRs and higher-end processors/DAC. It shows how much incoming jitter the device filters. 0 db means no filtering at all. Negative numbers show progressive amounts of filtering. So the largest negative number is best. The horizontal axis is the jitter frequency:

As you see the mass market AVRs all bunch up on top. Some even amplify jitter! That is a symptom of the peaking in the PLL loop filter.

Next best is my 10 year old Lexicon 12B processor. Much improved is the product from its sister company, the Mark Levinson 502 processor. These are both obsolete products yet outperform their much newer mass market AVRs from Onkyo, Pioneer, Yamaha and Anthem.

Producing absolutely superb performance is my ancient, circa 2000 Mark Levinson No 36S DAC.

What this means is that jitter in mass market devices is influenced by upstream devices/interconnect. So the performance I get with the same AVR may differ than you using the same AVR!

Clearly these techniques are not new. They just cost money and talent, something that is hard to come by in the money losing mass market AV business.

Jitter Listening test files

Reply #24 – 2014-12-10 15:31:12

Quote from: amirm on 2014-12-10 03:23:09

Quote from: MLXXX on 2014-12-10 03:06:57
I'm not privy to how phase lock loops audio streams are currently designed but I'd have thought it possible to have a coarse adjustment time constant if lock fails altogether such as when disconnecting and reconnecting an HDMI source, and a fine adjustment time constant once lock is achieved.

What you are describing is dual PLL solution which indeed accomplishes this goal. Alas, it is more expensive and complicated to design so you are going to see it in high-end gear, not mass market AVRs and such.

Here is a very revealing measurement I mad on a number of mass market AVRs and higher-end processors/DAC. It shows how much incoming jitter the device filters. 0 db means no filtering at all. Negative numbers show progressive amounts of filtering. So the largest negative number is best. The horizontal axis is the jitter frequency:

As you see the mass market AVRs all bunch up on top. Some even amplify jitter! That is a symptom of the peaking in the PLL loop filter.

Next best is my 10 year old Lexicon 12B processor. Much improved is the product from its sister company, the Mark Levinson 502 processor. These are both obsolete products yet outperform their much newer mass market AVRs from Onkyo, Pioneer, Yamaha and Anthem.

Producing absolutely superb performance is my ancient, circa 2000 Mark Levinson No 36S DAC.

What this means is that jitter in mass market devices is influenced by upstream devices/interconnect. So the performance I get with the same AVR may differ than you using the same AVR!

Clearly these techniques are not new. They just cost money and talent, something that is hard to come by in the money losing mass market AV business.

It is all just numbers, and in this case I'm not even seeing any absolute numbers, just relative numbers to some product I really know nothing about.

When it comes to detecting jitter by ear, Amir's ABX test results to date correspond to approximate -55 dB 30 Hz jitter, which is 30-40 dB worse than the worst commercial product that I know of. There are two steps of tougher tests in this Dropbox Folder:

https://www.dropbox.com/sh/b35feharwc7doty/...BTPbBa1JZa?dl=0