Looking at the way that bit reduction / dither noise is calculated for each of the dither options, it appears that I neglected to ensure that the rounded value remained within the permissible sample limits when calculating the noise from rounding and dithering. I have re-written my noise calculation subroutine and will revise the constants used in the code to recreate the dither noise surfaces (1..32 bits x 6..15 bit fft length x 3 dither options).
On the experimental spreading function front, I am looking at a spreading function which changes from averaging at small fft lengths to simple maximum at long fft lengths as follows:
begin
pcll:=low_frequency_bin[analysis_number]-1;
pchl:=high_frequency_bin[analysis_number]-1;
for pci:=0 to pchl-pcll+1 do
Begin
v1:=fft_result[pci];
v2:=fft_result[pci+1];
v3:=fft_result[pci+2];
vMax:=max(v1,max(v2,v3));
vMin:=min(v1,min(v2,v3));
vTot:=v1+v2+v3;
vMid:=vTot-vMax-vMin;
vAvg:=vTot/3;
Case fft_bit_length[analysis_number] of
0.. 6 : fft_result2[pci+1]:=(vAvg);
7 : fft_result2[pci+1]:=(vMax*1.50+vMid+vMin*0.5)/3;
8 : fft_result2[pci+1]:=(vMax*2.00+vMid)/3;
9 : fft_result2[pci+1]:=(vMax*2.50+vMid*0.5)/3;
10..15 : fft_result2[pci+1]:=(vMax);
End;
End;
end;