Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: AudioWorklet-based filter bank spectrum analyzer (Read 2069 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

AudioWorklet-based filter bank spectrum analyzer

Hi, I've have been written some JS code and posted this project "Analog-style spectrum analyzer and sliding DFT visualization using AudioWorklet" over CodePen, which I started out as testbed to figure out how to implement AudioWorklet-based stuffs (e.g. a type of real-time audio analysis like this not possible with AnalyserNode.getFloatTimeDomainData) and evolved into a spinoff of " Frequency bands spectrum analyzer using either FFT or CQT" project

Summary of this project:
  • Perform the sample acquisition on the AudioWorkletProcessor part similar to AnalyserNode.getFloatTimeDomainData but better (65536 or larger samples + ChannelSplitterNode-less multichannel analysis support perhaps). In this case, getting audio samples ranging from last samples since previous calculation to latest samples
  • Transfer the audio chunk into main thread through postMessage() and onMessage() function on each ports
  • Then in the main thread, before processing for realtime spectral analysis, downmix all channels into single mono (all channels incl. LFE, same as "Downmix channels to mono" DSP from foobar2000) audio chunk
  • Perform time-domain filter bank analysis (either analog-style IIR filter bank, sliding windowed infinite Fourier transform, or constant-Q/variable-Q sliding DFT)
  • Finally, plot the bargraph of the resulting spectrum data

Here's the screenshot of this visualization project (1/24th octave bands, color gradient similar to fb2k's built-in "Spectrum" visualization, 4th order analog-style filter bank):
X

Of course, you can also try analog-style and SWIFT filter bank analyzer on foobar2000 using Spectrum Analyzer (foo_vis_spectrum_analyzer) component, though any inquiries about this component should be posted on a thread about this foobar2000 component instead and any inquiries and questions about the project itself (e.g. performance and possible optimizations on VQ-sDFT) should be posted here

 

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #1
Features I currently have in this project:
  • 1/nth octave bands mode (equal-tempered scale, adjustable reference frequency and supports transposing like in old foo_musical_spectrum component for foobar2000) and frequency bands mode (supports Mel/Bark/ERB psychoacoustic scales and others like hyperbolic sine, nth root, and period, in-addition to standard linear and logarithmic frequency scales)
  • Adjustable frequency range in Hz for frequency bands equally spaced in arbitrary frequency scale, and in note index # (starting from C0 = ~16Hz) on octave bands mode
  • Truly constant-Q (assuming logarithmic frequency scale since the bandwidth is automatically determined by frequency band's lower and upper boundaries), no longer limited to 32768 samples (~682.7ms at 48kHz sampling rate) but it can be set to be variable-Q by unchecking "Use constant-Q instead" if you want better time resolution at lower frequencies, where minimum Hz resolution (to cope with time/frequency resolution tradeoff) is specified by "time resolution" parameter in milliseconds
  • Bandwidth for this filter bank is adjustable; higher values makes a smoothed spectrum while having improved time resolution on bass frequencies at the same time and vice versa
  • Three filter bank types are supported:
    • Analog-style analyzer: Cascaded biquad bandpass filters, simply stacking one another is good enough even without a flat-top response of Butterworth bandpass filters. Also, Q values are prewarped (unless you opt-out) for "truly" logarithmic resolution as this filter bank is designed using bilinear transform (borrowed from RBJ EQ cookbook)
    • Sliding windowed infinite Fourier transform: which is a bank of IIR complex resonators and it resembles a Gammatone filter on 4th order cascaded SWIFT
    • Variable-Q sliding DFT: Recursive FIR filter bank, complex-valued. Also have an option to use NC method to enhance time/frequency resolution. Window function options are limited to cosine sums like Hann and Hamming windows but custom frequency-domain windowing are supported and since it is a FIR type of filter bank, there is a "maximum time resolution" parameter that determine how long the circular buffer should be, which is no longer constant-Q if gets too low
  • For IIR filter bank modes: Adjustable filter order, higher values reduces leakage at the expense of some time resolution (especially on lower frequencies)
  • Peak decay and exponential moving average-based smoothing as well as optional fading peaks effect. Also an option to perform time smoothing operation during calculation rather than after (in other words, per-sample instead of per-frame) for greater accuracy and framerate-independence
  • Linear and nth root amplitude scale is supported in-addition to logarithmic/dB scale. Also, dB range on this visualization can be adjusted and "Use absolute value" sets minimum dB range to -∞ dBFS on linear/nth root amplitude mode
  • Multiple X-axis scales supported:
    • Decade: A standard grid for logarithmic scale
    • Octaves: Each label/line corresponds to center frequencies of a common 10-band graphic equalizer
    • Notes: Frequency gridlines are equidistantly-spaced in logarithmic frequency scale, following 1/12th octave bands and each label displays musical notes instead of Hz
    • Automatic: Frequency grid-lines and labels corresponds to actual frequency bands, much like foobar2000's built-in "Spectrum" visualization though X-axis labels become cluttered on large number of bands
  • Y-axis labels are in dB even if in linear/nth root scale and dB step for Y-axis grid are adjustable. Common dB interval values to set are 6dB, 10dB, 12dB, and 20dB, though it can be almost any value
  • As this project is intended towards audio analysis algorithms (in this case, IIR filter banks) and not intended to be eye-candy after all, color customization is very limited; it boils down to just light/dark mode and a switch between solid color and color gradient (borrowed from foobar2000 built-in visualizations for gradients)

This works best if you have a CPU that is much newer than a relatively-ancient Intel Core 2 Duo series as it demands more CPU than FFT-based analyzers using AnalyserNode.getFloatTimeDomainData() + FFT library

BTW, when using "Analog-style analyzer" mode and a "filter order" of 1, it produces a shape of the frequency bars visualization that reminds me of a hybrid of Windows Media Player's "Bars" and VLC media player's "Spectrum" visualizations where the former is for true log frequency scale and general shape, and the latter for spectral leakage and apparently equal peak width

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #2
Planned features to add:
  • Infinite average spectrum (which is what one of many things I wanted to be added in version 2.x.x.x of @Crossover's foo_enhanced_spectrum_analyzer component for foobar2000) like this:
    X
    X
  • Bark scale labels and grid, which follows more closely to auditory perception (I already know that it is neither linear nor logarithmic) than standard octave interval found in typical graphic equalizer effects from media players like Winamp. Something like this:
    X
  • Listen to certain frequencies feature like in Voxengo SPAN, which alters the audio output to what filter bank output sound like but it won't affect the analyzer (BTW, this is not possible as foobar2000 component iirc, but it is definitely possible as a VST plugin). Requires a new separate AudioWorkletProcessor dedicated for that since the filter bank analyzer is actually calculated in main thread (much like "Visualize with VST" feature in foo_dsp_vst3 component)
  • Tooltips on hovering over the visualizer window, to show Hz, closest note to frequency, and what dB corresponds to cursor, similar to foo_musical_spectrum

Once I have been able to login into CodePen, I would also make the mockups for future foobar2000-related stuffs (especially visualization and DSP components, including new features and proposed changes for existing components like Enhanced Spectrum analyzer) more public (through CodePen projects, which serves as an interactive mockup as usual) in-addition to implementing these proposed features above

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #3
Sorry but this all thing is just pseudo-science. Above generated spectrums do not match reality and are extremely low resolution.
Please remove my account from this forum.

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #4
Sorry but this all thing is just pseudo-science. Above generated spectrums do not match reality and are extremely low resolution.
except that analog audio spectrum analyzers (including but not limited to MSGEQ7) and some digital ones like the spectrum analyzers in display of RME DACs (which I don't have) do use IIR filter bank instead of FFT, though usually are "low-resolution" 1/3rd octave bands or full-octave, but some of them like Spectralissime do go up to 1/24th octave resolution and not using FFT and it have advantages over FFT like an ability to set time/frequency resolution tradeoff per-band (which mean you can have precise frequency resolution on bass frequencies and faster time resolution on higher frequencies at the same time) and change the frequency bin distribution to something like logarithmic frequency scales like octave bands

BTW, what about 1/48th octave spectrum (6th order cascaded biquads) and 1/192th octave bands (not cascaded, and it is not implemented in this project)? Satisfied?
X
X