Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: AudioWorklet-based filter bank spectrum analyzer (Read 4851 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

AudioWorklet-based filter bank spectrum analyzer

Hi, I've have been written some JS code and posted this project "Analog-style spectrum analyzer and sliding DFT visualization using AudioWorklet" over CodePen, which I started out as testbed to figure out how to implement AudioWorklet-based stuffs (e.g. a type of real-time audio analysis like this not possible with AnalyserNode.getFloatTimeDomainData) and evolved into a spinoff of " Frequency bands spectrum analyzer using either FFT or CQT" project

Summary of this project:
  • Perform the sample acquisition on the AudioWorkletProcessor part similar to AnalyserNode.getFloatTimeDomainData but better (65536 or larger samples + ChannelSplitterNode-less multichannel analysis support perhaps). In this case, getting audio samples ranging from last samples since previous calculation to latest samples
  • Transfer the audio chunk into main thread through postMessage() and onMessage() function on each ports
  • Then in the main thread, before processing for realtime spectral analysis, downmix all channels into single mono (all channels incl. LFE, same as "Downmix channels to mono" DSP from foobar2000) audio chunk
  • Perform time-domain filter bank analysis (either analog-style IIR filter bank, sliding windowed infinite Fourier transform, or constant-Q/variable-Q sliding DFT)
  • Finally, plot the bargraph of the resulting spectrum data

Here's the screenshot of this visualization project (1/24th octave bands, color gradient similar to fb2k's built-in "Spectrum" visualization, 4th order analog-style filter bank):
X

Of course, you can also try analog-style and SWIFT filter bank analyzer on foobar2000 using Spectrum Analyzer (foo_vis_spectrum_analyzer) component, though any inquiries about this component should be posted on a thread about this foobar2000 component instead and any inquiries and questions about the project itself (e.g. performance and possible optimizations on VQ-sDFT) should be posted here

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #1
Features I currently have in this project:
  • 1/nth octave bands mode (equal-tempered scale, adjustable reference frequency and supports transposing like in old foo_musical_spectrum component for foobar2000) and frequency bands mode (supports Mel/Bark/ERB psychoacoustic scales and others like hyperbolic sine, nth root, and period, in-addition to standard linear and logarithmic frequency scales)
  • Adjustable frequency range in Hz for frequency bands equally spaced in arbitrary frequency scale, and in note index # (starting from C0 = ~16Hz) on octave bands mode
  • Truly constant-Q (assuming logarithmic frequency scale since the bandwidth is automatically determined by frequency band's lower and upper boundaries), no longer limited to 32768 samples (~682.7ms at 48kHz sampling rate) but it can be set to be variable-Q by unchecking "Use constant-Q instead" if you want better time resolution at lower frequencies, where minimum Hz resolution (to cope with time/frequency resolution tradeoff) is specified by "time resolution" parameter in milliseconds
  • Bandwidth for this filter bank is adjustable; higher values makes a smoothed spectrum while having improved time resolution on bass frequencies at the same time and vice versa
  • Three filter bank types are supported:
    • Analog-style analyzer: Cascaded biquad bandpass filters, simply stacking one another is good enough even without a flat-top response of Butterworth bandpass filters. Also, Q values are prewarped (unless you opt-out) for "truly" logarithmic resolution as this filter bank is designed using bilinear transform (borrowed from RBJ EQ cookbook)
    • Sliding windowed infinite Fourier transform: which is a bank of IIR complex resonators and it resembles a Gammatone filter on 4th order cascaded SWIFT
    • Variable-Q sliding DFT: Recursive FIR filter bank, complex-valued. Also have an option to use NC method to enhance time/frequency resolution. Window function options are limited to cosine sums like Hann and Hamming windows but custom frequency-domain windowing are supported and since it is a FIR type of filter bank, there is a "maximum time resolution" parameter that determine how long the circular buffer should be, which is no longer constant-Q if gets too low
  • For IIR filter bank modes: Adjustable filter order, higher values reduces leakage at the expense of some time resolution (especially on lower frequencies)
  • Peak decay and exponential moving average-based smoothing as well as optional fading peaks effect. Also an option to perform time smoothing operation during calculation rather than after (in other words, per-sample instead of per-frame) for greater accuracy and framerate-independence
  • Linear and nth root amplitude scale is supported in-addition to logarithmic/dB scale. Also, dB range on this visualization can be adjusted and "Use absolute value" sets minimum dB range to -∞ dBFS on linear/nth root amplitude mode
  • Multiple X-axis scales supported:
    • Decade: A standard grid for logarithmic scale
    • Octaves: Each label/line corresponds to center frequencies of a common 10-band graphic equalizer
    • Notes: Frequency gridlines are equidistantly-spaced in logarithmic frequency scale, following 1/12th octave bands and each label displays musical notes instead of Hz
    • Automatic: Frequency grid-lines and labels corresponds to actual frequency bands, much like foobar2000's built-in "Spectrum" visualization though X-axis labels become cluttered on large number of bands
  • Y-axis labels are in dB even if in linear/nth root scale and dB step for Y-axis grid are adjustable. Common dB interval values to set are 6dB, 10dB, 12dB, and 20dB, though it can be almost any value
  • As this project is intended towards audio analysis algorithms (in this case, IIR filter banks) and not intended to be eye-candy after all, color customization is very limited; it boils down to just light/dark mode and a switch between solid color and color gradient (borrowed from foobar2000 built-in visualizations for gradients)

This works best if you have a CPU that is much newer than a relatively-ancient Intel Core 2 Duo series as it demands more CPU than FFT-based analyzers using AnalyserNode.getFloatTimeDomainData() + FFT library

BTW, when using "Analog-style analyzer" mode and a "filter order" of 1, it produces a shape of the frequency bars visualization that reminds me of a hybrid of Windows Media Player's "Bars" and VLC media player's "Spectrum" visualizations where the former is for true log frequency scale and general shape, and the latter for spectral leakage and apparently equal peak width

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #2
Planned features to add:
  • Infinite average spectrum (which is what one of many things I wanted to be added in version 2.x.x.x of @Crossover's foo_enhanced_spectrum_analyzer component for foobar2000) like this:
    X
    X
  • Bark scale labels and grid, which follows more closely to auditory perception (I already know that it is neither linear nor logarithmic) than standard octave interval found in typical graphic equalizer effects from media players like Winamp. Something like this:
    X
  • Listen to certain frequencies feature like in Voxengo SPAN, which alters the audio output to what filter bank output sound like but it won't affect the analyzer (BTW, this is not possible as foobar2000 component iirc, but it is definitely possible as a VST plugin). Requires a new separate AudioWorkletProcessor dedicated for that since the filter bank analyzer is actually calculated in main thread (much like "Visualize with VST" feature in foo_dsp_vst3 component)
  • Tooltips on hovering over the visualizer window, to show Hz, closest note to frequency, and what dB corresponds to cursor, similar to foo_musical_spectrum

Once I have been able to login into CodePen, I would also make the mockups for future foobar2000-related stuffs (especially visualization and DSP components, including new features and proposed changes for existing components like Enhanced Spectrum analyzer) more public (through CodePen projects, which serves as an interactive mockup as usual) in-addition to implementing these proposed features above

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #3
Sorry but this all thing is just pseudo-science. Above generated spectrums do not match reality and are extremely low resolution.
Please remove my account from this forum.

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #4
Sorry but this all thing is just pseudo-science. Above generated spectrums do not match reality and are extremely low resolution.
except that analog audio spectrum analyzers (including but not limited to MSGEQ7) and some digital ones like the spectrum analyzers in display of RME DACs (which I don't have) do use IIR filter bank instead of FFT, though usually are "low-resolution" 1/3rd octave bands or full-octave, but some of them like Spectralissime do go up to 1/24th octave resolution and not using FFT and it have advantages over FFT like an ability to set time/frequency resolution tradeoff per-band (which mean you can have precise frequency resolution on bass frequencies and faster time resolution on higher frequencies at the same time) and change the frequency bin distribution to something like logarithmic frequency scales like octave bands

BTW, what about 1/48th octave spectrum (6th order cascaded biquads) and 1/192th octave bands (not cascaded, and it is not implemented in this project)? Satisfied?
X
X


 

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #6
New feature: Infinite average spectrum
  • Can use either of three different averaging domains:
    • Linear: Good approximation to RMS averaging if you don't care about mathematical definition
    • Squared (RMS): Follows the exact mathematical definition of root mean square (RMS) so it is my personal favorite domain for averaging the spectrum and the RMS part of peakmeter visualizations
    • Logarithmic: Similar to foo_enhanced_spectrum_analyzer's current logarithmic averaging but converging
  • Can be frozen (but not the main spectrum nor the peaks) separately where as "Freeze analyzer" stops audio analysis altogether

It would look like this:
X

In-addition to that, the analyzer can optionally be reset automatically when seeking (roughly similar to foobar2000 visualizations like Enhanced Spectrum analyzer component), needed for infinite averaging to work well as if not reset when playing a new song, the infinite average part might stay the particular curve from previous song

BTW, I made an another project (spun off from this one) about an AudioWorklet-based multi-channel peakmeter, which is currently in WIP (working-in-progress) state so no Mid/Side representations yet (as in previous AnalyserNode-based one) on this newly-made project in the meantime

BTW, this new project would look like this below:
For mono:
X
For stereo (which is the most common channel configuration for audio files I've seen AFAIK):
X
For multichannel (something like 5.1 surround sound):
X

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #7
New feature: Draw line/area graph instead of bars
X
Note that fading peaks don't work on linegraph mode (though it is possible for @Crossover to make that happen for foo_enhanced_spectrum_analyzer 2.x.x.x) and it uses linearly-interpolated curves instead of smoother quadratic/cubic/bezier spline curves as in foo_vis_spectrum_analyzer by @pqyt

Also the configuration settings got changed; moved appearance-related settings to its own "Appearance" section

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #8
New feature: Customizable frequency bars width and spacing options, borrowed from my another project. Note that "pixel perfect" spacing mode (which should emulate non-antialiased bargraph spectrum) currently introduces "blur" when the bar gap width is either an odd number or noninteger value

And also the new frequency grid options are carried over from the recent update of this FFT-based spectrum analyzer and spectrogram visualization project, quoting this:
Also, new frequency grid modes:
  • Coarser decadic gridlines: (top is similar to current foo_enhanced_spectrum_analyzer's frequency label layout, bottom is without minor labels/gridlines)
  • Power of two labels and gridlines where every label is the exact center frequency of C note when C5 = 512Hz:
    This is similar to standard octave band gridlines (which lines up with standard graphic equalizer effect in audio players like Windows Media Player), which look like this:

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #9
Finally, I've updated this project with:
  • NEW: First in, first out (FIFO) averaging, which can even be combined with infinite average graph. Note, this type of averaging uses of a lot of CPU, which could mean this part is the one where using AVX2 instructions is helpful for foobar2000 components like foo_enhanced_spectrum_analyzer right? And you need a beefy CPU to run FIFO average graph at 60fps
  • NEW: Spectrogram display (and combined spectrum/spectrogram display where bottom part acts like waterfall), with adjustable "hop size" (in samples) to change spectrogram scrolling speed
  • NEW: Alternate color scheme (on color gradient mode, it changes into something similar to foo_enhanced_spectrum_analyzer's default color preset, and on solid color mode, it changes the foreground and background color of dark mode into something reminiscent of foobar2000's built-in visualizations on dark mode)
  • NEW: Linear frequency grid/label mode (works best on linear frequency scale)
  • ENHANCEMENT: Main graph can be hidden (e.g. only show FIFO average graph for highly-accurate replica of Monstercat visualizer, which is something for foobar2000 users wanting Monstercat-like visualizer on their Columns UI layout through foo_enhanced_spectrum_analyzer 2.x.x.x perhaps)

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #10
New update: Added a feature to decouple amplitude scaling of spectrogram from spectrum display (which is especially for combined spectrum/spectrogram)

BTW, the spectrogram scrolling speed (determined by "hop size") only works when "Apply time smoothing during processing" is turned on

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #11
Will it be hardware accelerated in foobar2000 as the default ones are?
TAPE LOADING ERROR

Re: AudioWorklet-based filter bank spectrum analyzer

Reply #12
Will it be hardware accelerated in foobar2000 as the default ones are?
Probably yes with Advanced Vector Extensions 2 (AVX2) instructions for calculation of spectral analysis and averaging algorithms (which should allow 1920 bands filter bank analyzer to run at consistent 60fps) and GPU acceleration for rendering graphical stuffs (given the @Crossover's upcoming foo_enhanced_spectrum_analyzer 2.x.x.x ditches GDI+ in favor of OpenGL to keep up with hardware-accelerated version of foobar2000's built-in visualizations), in-addition to multithreading for multiple channels visualization (e.g. Mid/Side spectrum analyzer and 5.1 surround sound analyzer) where each channel has its own thread for spectrum calculations, in a foobar2000 spectrum analyzer component

BTW, this project has been updated to expand the customization of spectrum drawing; three drawing modes (stroke, fill, or both) for each three type (main, peak, average/RMS), which is derived from my FFT-based audio visualization project