Skip to main content
Topic: Proper way to perform a null test (Read 9604 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Proper way to perform a null test

What is the best/recommended way to perform a null test between two audio tracks? What program to use? How to exactly align the tracks in time?

Any advice or help is welcome. Thanks.

Proper way to perform a null test

Reply #1
Almost any audio editor can be used.

Proper way to perform a null test

Reply #2
One track must be inverted.
The two tracks must be mixed together
or
one copied and pasted over the other.
The only way I know to assure proper alignment is to zoom in to the individual sample level and identify a common starting point in each track. Being off just one sample can make a big difference.


Proper way to perform a null test

Reply #4
That sounds like a pretty neat, but frequently impossible, trick.

Proper way to perform a null test

Reply #5
The null test is one of my favorite tests. It proves that two audio streams are identical when the difference signal is exactly zero. Perfect nulling is only possible in the digital domain.
If the difference signal isn't zero, all bets are off and all you can do is make an educated guess by listening to or looking at the difference signal.
There can be 3 scenarios:
a) perfect null, the audio streams are identical and should (will!) sound identical. No more listening is necessary.
b) non-perfect null, the audio streams are not identical, but perceptually indistinguishable (until you find a person who can hear a difference).
c) non-perfect null, the audio streams are not identical and perceptually different (ABX-able).

Like the two others said, you need an audio editor (like Reaper), capable of mixing 4 tracks into 2.
-Open file A in tracks 1+2 and file B in tracks 3+4.
-Move one of them until they are perfectly sample-accurately aligned.
-Invert the polarity (phase) of A or B.
-Play all 4 tracks. Listen and watch the stereo output master peak meters. With a bit a luck they won't move. You can move up the master fader to its max to verify there's only silence. SET IT BACK TO ZERO RIGHT AWAY TO PROTECT YOUR EARS AND EQUIPMENT !

If the output meters still move it could be that the files are:
1) not perfectly aligned. Move sample by sample until the output is as low as possible (hopefully -inf.)
2) not the same level. Adjust the level of A or B until the mixer output is as low as possible. Mixing desk faders might not be accurate enough though.
3) simply not identical. In some cases the difference is white noise (dither?), or only certain frequencies, when one of the files has been EQ'd.
4) not sync because the timing difference between A and B is less than one sample. This can e.g. happen with SRC (sample rate conversion). Mac users can try a handy Audio Unit plugin which allows for sub-sample delays (airwindows.com).

If the output isn't zero it can be handy to record the mix to a file for closer examination (like FFT). Make sure that the output format is suitable (24 or 32 bits is fine) and that no processing (like src or dither) is applied.

Hope this helps.
Kees de Visser

Proper way to perform a null test

Reply #6
Some other things to add:

1. Be careful how you measure the output, as some programs won't show anything below a certain level. For instance I know Sonar's meters show -∞ below ~-100dBFS even if there is a signal there. In this case it is not a problem for the kind of usage the program is intended for, but is if you're looking for a "perfect null".

2. If you are comparing things like tracks or exports in a DAW, be aware that many things like VST processors or synths contain random processing that will differ on each export. To check for this, you can export the exact same thing twice without changing whatever you're testing for and verify that they null perfectly before proceding.

Proper way to perform a null test

Reply #7
Quote
How to exactly align the tracks in time?
If you have to re-align the tracks, there's a difference and you have already "failed" the null test. 

The most important thing to remember is that the sound of the difference is NOT the same as the difference in the sound!  Two files can sound identical when there are gross differences that show-up in a null test.

There are at least 3 ways to demonstate this:

1. Starting with two bit-identical files, time-delay one copy by a few milliseconds - There is no difference in the sound, but the subtracted difference proves there is a huge difference, and the difference file sounds nothing like a delay.  (If you have experience with this stuff, you might recongize it as the side-effect of a delay.)

2. Take two bit-identical files and invert one copy - Again no difference in sound.  The difference-file (now "subtracting a negative")  sounds identical to the original files, except louder.  Again proving a huge difference in the "data" with no difference in sound.

3. Perform a null-test between two completely different-unrelated files.  Now the difference-file is simply a mix of the two files that sounds identical to summation of the two files.    This could even be two different recordings of the same song, say a live and a studio version, or the same song by two different artists...  The null isn't going to tell you anything about the difference between the two versions, you'll just hear both versions mixed.



...I'm NOT saying that null tests are worthless.  If you get silence (or a very quiet file) there is no difference (or very little difference) in sound.  However, it doesn't work the other way around.

Proper way to perform a null test

Reply #8
If you have to re-align the tracks, there's a difference and you have already "failed" the null test.
Nonono, forget about comparing files. That's too easy with a computer
The great advantage of a null test in a DAW (editor) or any other mixer is that you can compare audio streams of different duration, bit depth and even sampling rate (if you accept that upsampling is near-transparent). It's even possible with analog sources, like 2 tape machines, but the null will never be perfect in the analog domain. In this case the null test is used to synchronize two sources, not to proof similarity. I've used this a lot to sync my DAW to an external video machine's audio track.

Proper way to perform a null test

Reply #9
I've done a little null testing with a DXD (24 bit 352.8 kHz) source (Haydn from the 2L.no site) and a self made 44.1 version. It can be shown that the difference is at a very low level, about 15 dB below 24-bit dither. So if hi-res formats have any benefits, it has to be the extended bandwidth above 20kHz, because below 20 kHz there is no difference to speak of (at least not in the digital domain).



Proper way to perform a null test

Reply #10
Quote
If you have to re-align the tracks, there's a difference and you have already "failed" the null test.
Many times a file will contain some samples at the beginning or the end that were not in the original. This is probably the norm when extracting from CD. Perhaps it has to with offsets. Regardless, these are always either zero values or very low level noise, but the principal would be the same if it were a burst of very loud static or a curse from Satan. Or, maybe a few bits will be lost instead of added.

Even so, the audio data of interest may be, and generally will be, barring damaged media, bit for bit identical with the original source to which one wants to compare. However, if the correct alignment is not achieved before adding the two files, the results will be very different than the string of zero values that mean the two are identical.

If you want to argue that the two were not identical because of the added or missing bits at the end point(s), you are free to live in that universe, but in most cases you are not obtaining very useful information.

If the answer to 'how to align' is still open, this is sometimes difficult.  If the goal is to find out if one can extract accurately, one can create test data by
pulling up one sample (say 20dB) near the beginning of each track (or pulling down, if the track opening is very loud), creating a very easy to identify sample on which to align
writing these modified tracks to CD-R,
extract them from the CD-R,
then run the test on each extracted against source track.

If one has tracks where some particular beginning sample is too difficult to identify, one can make a best guess and do the test. If the result is not nulled, one can move one of the tracks by one bit and try again. This can be very tedious if one is using too vague a starting point, but it can work if one is persistent.

Best in this case is to go to some identifiable samples to find a start point. It doesn't matter if this means ignoring the first second or two, or whatever. If one get null values by starting at this identified place, one can then work backwards to find out when the tracks begin to differ. A good audio editor will show the sample count. Thus one can easily step forward or backwards by 151,329 samples, or any other particular number, allowing a more reasonable search procedure.


Proper way to perform a null test

Reply #11
[quote author=AndyH-ha link=msg=863356 date=1397773845]This can be very tedious if one is using too vague a starting point, but it can work if one is persistent.[/quote]Another option is to insert a sample-accurate delay plugin into one of the stereo pairs, play both pairs and adjust the delay on the fly until both streams are perfectly sync. With a bit of training this can be done within a minute. When sync is far off you hear double sources, like an echo. When sync is close, the mix becomes colored (comb filtering) and when sync is perfect (within one sample) the mix output will be silence or very low level, depending on how identical the two sources are.

Proper way to perform a null test

Reply #12
I used a similar trick back in analogue tape days. I wired up a short headphone extension lead with the ground connection lifted. I'd play a test tape, or the tape which I needed to align the deck to, and adjust the azimuth for minimum output. The advantage was that it worked with normal program material and was quicker than and as accurate as peaking for maximum HF.
Regards,
   Don Hills
"People hear what they see." - Doris Day

Proper way to perform a null test

Reply #13
Adobe Audition has fairly easy to use 'invert /mixpaste' option that's perfect for null tests.  (You still have to do alignment yourself though, if it needs doing).



Adobe was *giving away* Audition awhile back on its site...don't know if that's still true.

Proper way to perform a null test

Reply #14
Adobe Audition has fairly easy to use 'invert /mixpaste' option that's perfect for null tests.  (You still have to do alignment yourself though, if it needs doing).


Be aware that the above works, BUT an apparent alternative does not: You should not invert one track first as a separate operation, because the inversion result is stored as 16-bits and full scale 16-bit samples cannot always be inverted: values of -32768 will be converted to +32767, because +32768 does not exist. (the number code space is asymmetrical, to leave space for zero)

Hence if you compare bit-identical clipped audio tracks by inverting one separately first, then when you add it to the non-inverted track, it will not perfectly null, but will leave differences with an amplitude of one least significant bit.

Cheers,
David.

Proper way to perform a null test

Reply #15
The function is actually called just 'mixpaste', but it includes the option to check an 'invert' box.

Proper way to perform a null test

Reply #16
You should not invert one track first as a separate operation, because the inversion result is stored as 16-bits and full scale 16-bit samples cannot always be inverted: values of -32768 will be converted to +32767, because +32768 does not exist. (the number code space is asymmetrical, to leave space for zero)
Further to the discussion of this held here previously, I’m wondering if there were ever any rough stats on how many recordings ever actually use that lowest value? I completely understand why 2’s-complement is used in computing, but it seems to me the best idea is to eschew use of the most negative number, limiting that side to the same absolute maximum as the positive. Parameter scales that are asymmetrical for this reason really offend my desire for neatness!  And I guess now I’m off to search for philosophical debates about this number

Proper way to perform a null test

Reply #17
You should not invert one track first as a separate operation, because the inversion result is stored as 16-bits and full scale 16-bit samples cannot always be inverted: values of -32768 will be converted to +32767, because +32768 does not exist. (the number code space is asymmetrical, to leave space for zero)

Interesting, I've never encountered that problem, so I did a quick test:
-Create a full-scale white noise 16/44.1 aiff file.
-Increase level 3 dB to create plenty of overloads and save file (still 16/44.1)
-Create a phase inverted copy (16/44.1)
-Mix both files (1:1) and save as new file (16/44.1)

The difference file contains no signal.

My practice doesn't seem to agree with your theory. Any idea why ?

Proper way to perform a null test

Reply #18
You start with 65536 possible values, and you end with 65536 possible values, so there is a 1:1 mapping that will work.

Namely, if you perform a 1's complement then +32767 becomes -32768, -32768 becomes +32767, 0 becomes -1 and -1 becomes 0, etc.

I don't know if this is how the problem is generally avoided, but it certainly would work.

Proper way to perform a null test

Reply #19
No, the problem is avoided by doing these operations at higher bit depths. Audition does this automatically.

0 = 0, (-1)*1 = -1, (-1)*32767 = -32767 and (-1)*-32768 = clipped to 32767.

In Audition you need to save both normal and inverted as 16-bit files and reopen them, then mix-paste and you'll see it will only null to -1 on the negative side.
(to make sure the inverted file isn't dithered on save create it as 24+ bit, paste, invert, convert it to 16 bit without dither, then save)
"I hear it when I see it."

Proper way to perform a null test

Reply #20
So, the last few posts are with, or against 2BDecided's post?

From the way they are redacted, it is not clear. From the implications, It seems obvious that he is correct:

Case A) Program does 2's complement (binary inversion).  32767 becomes -32768, 0 becomes -1, 1 becomes 0.    0 + -1 = -1 , not 0,  32767 + -32768 becomes -1.
Case C) Program does 2's complement and clips:  32767 becomes -32767, 32766 becomes -32767, 0 becomes -1, 1 becomes 0. 0 + -1 = -1 32767 + -32767 = 0.
Case C) Program invert sign (decimal inversion) and clips. 32767 becomes -32767 becomes -32767, 32766 becomes -32766, 0 becomes 0, 1 becomes 1, -32768 becomes -32767.  0 + 0 = 0, 32767 + -32767 = 0. -32768 + 32767 = -1

In all the cases, one cannot be accurate up to the LSB (least significant bit). So, obviously, when converting 16bit to 24 bit and using case C, it works. In any of the other cases, it does not.

I repeat: invert paste, with integer, does not work down to the LSB.

Proper way to perform a null test

Reply #21
The function is actually called just 'mixpaste', but it includes the option to check an 'invert' box.
Yes, and that's the one that works.

The separate invert function does not. In CEP 1.2a anyway. I'm a Luddite.


You start with 65536 possible values, and you end with 65536 possible values, so there is a 1:1 mapping that will work.

Namely, if you perform a 1's complement then +32767 becomes -32768, -32768 becomes +32767, 0 becomes -1 and -1 becomes 0, etc.

I don't know if this is how the problem is generally avoided, but it certainly would work.
It would work for this specific use case, but more generally it would be strange to have an invert function that introduced a DC offset. I guess alternatively it's strange that the invert function potentially introduces (insignificant) clipping.

Aside from mathematical analysis, I can't imagine that anyone ever cared about these limitations.


You should not invert one track first as a separate operation, because the inversion result is stored as 16-bits and full scale 16-bit samples cannot always be inverted: values of -32768 will be converted to +32767, because +32768 does not exist. (the number code space is asymmetrical, to leave space for zero)
Further to the discussion of this held here previously, I’m wondering if there were ever any rough stats on how many recordings ever actually use that lowest value? I completely understand why 2’s-complement is used in computing, but it seems to me the best idea is to eschew use of the most negative number, limiting that side to the same absolute maximum as the positive. Parameter scales that are asymmetrical for this reason really offend my desire for neatness!
Me too. But in the context of the invention of the CD (i.e. pre DAW, pre audio in PCs etc), I guess none of this mattered. You would never have an absolutely zero DC offset in a practical A>D, and you would never be able to ensure that you just hit 32767 or 32768 - most early CDs had headroom between the maximum signal level and 0dB FS.

Cheers,
David.

Proper way to perform a null test

Reply #22
The function is actually called just 'mixpaste', but it includes the option to check an 'invert' box.
Yes, and that's the one that works.

The separate invert function does not. In CEP 1.2a anyway. I'm a Luddite.

Clearly then the difference is that it is OK to subtract -32768, but not to negate it first and then add it.

Proper way to perform a null test

Reply #23
[JAZ], the proper way to negate a 2's complement number is by flipping all bits and adding 1.
Since 32767+1 = -32768, there is no way to invert -32768.

I don't think programs internally operate with 16 bit integers, but more like 32 or even 64 bit ones (possibly even floating point), so you don't need any special inversion code.

I see no problem with invert, mix-paste. If your program fails this test you can always convert the files to a higher bit depth.
"I hear it when I see it."

Re: Proper way to perform a null test

Reply #24
Just tried to deliberately clip a file in Audition 1.5, it shows +/-32767
Then I clipped the file using foobar's file converter with replaygain's preamp and opened in Audition, it showed +32767/-32768
It seems that different software do this differently.

Now it becomes interesting. If I invert the +32767/-32768 file in Audition, the result is +/-32767!
Audition doesn't use -32768 unless I zoom the waveform to sample level and drag the dots.

 
SimplePortal 1.0.0 RC1 © 2008-2019