So I took this screenshot as reference to decide how much high frequency I should cut in my test file for this experiment:

In fields like this, we need a more scientific approach then blind ABX, adopting the so called "null test" (original signal + phase inverted reconstructed signal should produce silence) to understand the "AI" objective recontruction effectiveness....

I disagree: when the goal is to "reconstruct" or "enhance" the losses of a perceptual compression, the evaluation criterion should include some element of perception as well.

It could be argued that a traditional ABX (or ABXY) test is not the right test protocol, but I suspect that you can not get away with having to include some kind of subjective preference in the evaluation strategy.

Also, I am not familiar with the operation of Github, if someone can build a Windows executable without requiring specific GPU features I am willing to post some results, just like what I did previously.

I can see why there is still no one posted the results. Asking others for the processing results will deplete their cloud computing quota, and looks like I will need to learn some Python to automate the process as well, so I will give up.

The files were created by combining several existing tools/effects bundled with Reaper without using any additional plugins (e.g. iZotope, Thimeo Stereo Tool etc). However I am not going to describe the procedure in detail to avoid people using similar methods to generate large amount of fake files for ill purpose because the processing power requirement of this method is pretty low.

Just another observation of your null file. It is not level matched, it is 6dB higher than the fake88200.flac I posted.
Even though these kinds of null comparisons don't have too much psychoacoustic relevance, if one wants to do some null tests, at least do it honestly.

loaded 2 files in audacity, inverted the phase of the "fake" one and mixdown

Quote from: forart.eu on 2024-08-20 07:18:53

loaded 2 files in audacity, inverted the phase of the "fake" one and mixdown

Hm, what is then Audacity doing?

Hm, what is then Audacity doing?

DeltaWave v2.0.13, 2024-08-20T08:41:57.7713434+02:00
Reference: original.flac[L] 1323000 samples 44100Hz 16bits, stereo, MD5=00
Comparison: fake88200.flac[L] 2646000 samples 88200Hz 24bits, stereo, MD5=00
Settings:
Gain:True, Remove DC:True
Non-linear Gain EQ:False Non-linear Phase EQ: False
EQ FFT Size:65536, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -500dB
Correct Non-linearity: False
Correct Drift:True, Precision:30, Subsample Align:True
Non-Linear drift Correction:False
Upsample:True, Window:Kaiser
Spectrum Window:Kaiser, Spectrum Size:32768
Spectrogram Window:Hann, Spectrogram Size:4096, Spectrogram Steps:2048
Filter Type:FIR, window:Kaiser, taps:262144, minimum phase=False
Dither:False bits=0
Trim Silence:True
Enable Simple Waveform Measurement: False
Resampled Reference to 88200Hz
Discarding Reference: Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s
Initial peak values Reference: 0,082dB Comparison: 0dB
Initial RMS values Reference: -18,453dB Comparison: -18,394dB
Null Depth=34,424dB
Trimming 71424 samples at start and 0 samples at the end that are below -90,31dB level
X-Correlation offset: -32534 samples
Trimming 0 samples at start and 0 samples at the end that are below -90,31dB level
Drift computation quality, #1: Excellent (0,04μs)
Trimmed 7086 samples ( 80,340136ms) front, 23676 samples ( 268,435374ms end)
Final peak values Reference: 0,082dB Comparison: -0,115dB
Final RMS values Reference: -18,336dB Comparison: -18,366dB
Gain= 0,0923dB (1,0107x) DC=0,00029 Phase offset=-809,811296ms (-71425,356 samples)
Difference (rms) = -40,02dB [-41,64dBA]
Correlated Null Depth=45,45dB [52,48dBA]
Clock drift: -0,01 ppm
Files are NOT a bit-perfect match (match=0,16%) at 16 bits
Files match @ 49,9965% when reduced to 6,96 bits
---- Phase difference (full bandwidth): 25,074700939838°
0-10kHz: 16,85°
0-20kHz: 17,94°
0-24kHz: 20,38°
Timing error (rms jitter): 659,8ns
PK Metric (step=400ms, overlap=50%):
RMS=-43,0dBFS
Median=-44,8
Max=-34,5
99%: -35,57
75%: -42,43
50%: -44,82
25%: -47,33
1%: -53,32
gn=0,989435073400854, dc=0,000287857645362735, dr=-5,14181713836272E-09, of=-71425,3563045766
DONE!
Signature: 653fa7beb2363b5f7598785ac68bc85a
RMS of the difference of spectra: -94,9942219385772dB
DF Metric (step=400ms, overlap=0%):
Median=-24,9dB
Max=-21,7dB Min=-28,1dB
1% > -28,04dB
10% > -26,13dB
25% > -25,54dB
50% > -24,87dB
75% > -24,06dB
90% > -22,95dB
99% > -0,58dB
Linearity 2,1bits @ 0.5dB error
---- Phase difference (full bandwidth): 25,3354794968148°
0-10kHz: 12,28°
0-20kHz: 15,17°
0-24kHz: 18,99°
Linearity 2,1bits @ 0.5dB error

Even if I do the null as mentioned using Audacity, the resulting waveform still obviously different from his version. Something must be seriously screwed up if not intentional. Check out another video file attached.

Checked, the only two differences are that - as sayd - i've phase inverted the fake

Interesting that someone mentioned null test first without even paying attention to how do it in a sensible way.

Im torn on the notion that these algorithms cant recover lost information. I thought so too but this thread has had me thinking about it. After all they are making an educated guess, and that guess has a propability to be right, in which case it DID recover lost information. I think since we dont have the source material most of the time this question is more philosophical than anything.

DeltaWave v2.0.13, 2024-08-20T12:23:57.2062764+02:00
Reference: test_original_flac.flac[L] 7278467 samples 192000Hz 24bits, stereo, MD5=00
Comparison: test_converted_flac.flac[L] 8491546 samples 224000Hz 24bits, stereo, MD5=00
Settings:
Gain:True, Remove DC:True
Non-linear Gain EQ:False Non-linear Phase EQ: False
EQ FFT Size:65536, EQ Frequency Cut: 0Hz - 0Hz, EQ Threshold: -500dB
Correct Non-linearity: False
Correct Drift:True, Precision:30, Subsample Align:True
Non-Linear drift Correction:False
Upsample:True, Window:Kaiser
Spectrum Window:Kaiser, Spectrum Size:32768
Spectrogram Window:Hann, Spectrogram Size:4096, Spectrogram Steps:2048
Filter Type:FIR, window:Kaiser, taps:262144, minimum phase=False
Dither:False bits=0
Trim Silence:True
Enable Simple Waveform Measurement: False
Resampled Reference to 224000Hz
Discarding Reference: Start=0s, End=0s
Discarding Comparison: Start=0s, End=0s
Initial peak values Reference: -3,358dB Comparison: 0dB
Initial RMS values Reference: -20,25dB Comparison: -16,675dB
Null Depth=101,926dB
Trimming 0 samples at start and 0 samples at the end that are below -90,31dB level
X-Correlation offset: -3 samples
Trimming 0 samples at start and 0 samples at the end that are below -90,31dB level
Drift computation quality, #1: Excellent (0,3μs)
Trimmed 171244 samples ( 764,482143ms) front, 224860 samples ( 1003,839286ms end)
Final peak values Reference: -3,358dB Comparison: -3,801dB
Final RMS values Reference: -20,505dB Comparison: -20,765dB
Gain= 3,822dB (1,5528x) DC=0,00001 Phase offset=-0,013447ms (-3,012 samples)
Difference (rms) = -32,77dB [-39,68dBA]
Correlated Null Depth=42,37dB [41,89dBA]
Clock drift: -0,01 ppm
Files are NOT a bit-perfect match (match=0,13%) at 16 bits
Files are NOT a bit-perfect match (match=0%) at 24 bits
Files match @ 49,998% when reduced to 6,5 bits
---- Phase difference (full bandwidth): 143,874834932871°
0-10kHz: 23,92°
0-20kHz: 56,12°
0-24kHz: 66,07°
Timing error (rms jitter): 514ns
PK Metric (step=400ms, overlap=50%):
RMS=-37,5dBFS
Median=-46,7
Max=-29,4
99%: -30,89
75%: -34,7
50%: -46,7
25%: -50,74
1%: -55,33
gn=0,644017646514286, dc=1,40872942458951E-05, dr=-8,96573621136896E-09, of=-3,01221162486093
DONE!
Signature: 85634bda22fde7ebef2c2e8a0c038e7a
RMS of the difference of spectra: -78,1160326779584dB
DF Metric (step=400ms, overlap=0%):
Median=-18,9dB
Max=-10,6dB Min=-30,8dB
1% > -30,78dB
10% > -27,18dB
25% > -24,67dB
50% > -18,88dB
75% > -13,32dB
90% > -12,28dB
99% > -2,19dB
Linearity 2,1bits @ 0.5dB error

Quote from: bennetng on 2024-08-20 10:05:23

Interesting that someone mentioned null test first without even paying attention to how do it in a sensible way.

So we agree that the null test is the way to go, in this field.

At some point I may consider getting a new GPU and learn some Python then run the code locally on my PC, and do some listening tests myself.