Questions about visualization streams

2023-05-27 10:22:08

I have a question regarding visualization streams, especially with get_chunk_absolute(), since the sliding DFT only works if the audio data is fed into the algorithm properly (new data since last calculation)

Does get_chunk_absolute() have requested_length/2 offset ahead actual playback, or perhaps full samples offset ahead of it?

Re: Questions about visualization streams

Reply #1 – 2023-06-29 10:58:33

Apologies if my question sounds stupid. By looking at the specified line of unrelated component (foo_r128meter), it seems like the offset from actual playback (time of the latest sample from visualization streams vs the actual playback time in samples) is proportional to the requested length (not half of it contrary to oscilloscope visualization) right?

Especially when comes to something like an analog-style spectrum analyzer (By analog-style, I meant it doesn't use FFT, and instead it uses banks of IIR filters) inside foobar2000 (which doesn't exist yet besides VST plugins, which is out-of-scope for this topic), and when either the buffer alignment is wrong or the size of current visualization chunk is different from time difference between current and previous calculations in samples, it produces garbage results (where the frequency spectrum starts to look like spectrum of audio with clicking/popping caused by buffer underruns) as said in the comment of my non-realtime spectrogram sketch code (apologies if my sketch is completely unrelated to fb2k at all) when using either SWIFT or standard Sb-sDFT. So you need to have both correct alignment and buffer size (equal to time difference between current and previous like I said) to only cover newest samples since previous calculations in order to correctly visualize sliding DFT algorithm.

Re: Questions about visualization streams

Reply #2 – 2023-10-22 03:57:01

Another questions especially when comes to developing a Mid/Side spectrum visualization component:

• Does get_spectrum_absolute(chunk, offset, fftSize) returns real/imaginary parts or just magnitude data (I'd assume it is former since fb2k's built-in spectrum (and spectrogram with channel mode set to mono) doesn't react when the audio source is 180 degrees out of phase between left and right channels)? Which is relevant for doing M/S encode after FFT since doing it on magnitude-only FFT data would be improper M/S (as the former part doesn't get lower or zero when encountering the out-of-phase stereo signals, and the latter part is just a multiband L/R balance meter), and in some other cases like doing Lanczos FFT bin interpolation on complex FFT output (which look more like zero-padded spectrum than sinc/Lanczos interpolated magnitude-only FFT)
• And if not, are there a built-in way to get M/S spectrum data without having to do it myself (M = (L+R)/2 and S = (L-R)/2 respectively) by doing a M/S encode on channel pairs before doing an FFT of transformed signal?

Since I have no idea on how get_spectrum_absolute works beyond the fact it provides FFT data because I haven't used foobar2000 SDK yet (also, I can't find the information whether or not the get_spectrum_absolute() function returns complex FFT data) and I'm used to develop WebAudio visualizations like this that involves custom FFT implementation (this should also apply to fb2k component development as well as I think the enhanced spectrum analyzer already use custom FFT right?)

BTW, I've attached the audio-only .webm file (sure, foobar2000 can load webm files, except audio-only) of some sort of test tone thing that contains left-only, right-only, in-phase, and out-of-phase segments.