HydrogenAudio

This compressor is fully THATcorp/dbx style RMS, and has 3 bands:  20-160, 160-2600, 2600-20k.
Sounds much better when there is lots of bass, etc.

It does any compression ratio between 1:1 to inf:1.  The command line args are:

--cratio=xxx
Real compression ratio is 1/(1-cratio), where if cratio = 0.90, then the real compression ratio is 10:1

--thresh=xxx
Pivot location -- 9dB is a good value.

--ingain=xxx, --outgain=xxx
input/output gains in dB

Lots of things can change if you modify the source -- all of it is in there.

repo:  https://spaces.hightail.com/space/tjUm4ywtDR
Repo will contain examples in a while, and will have 'MB' in the name for 'multiband'.
John

Since there have been some downloads -- I will be posting some changes/updates to tweak the performance. As it is -- it is a bit 'fast'.   I have really offered this for people who want to work on the source and change its behavior -- but I'll be tweaking it over time.

THIS IS BRAND NEW CODE -- in the sense that the compressor has only existed for a few hours of working on it.  I have a few good updates coming.  Also -- I'll post the source changes so you can see the progress.

right now, it is happiest runnjing at 1.5:1 or lower, but can sometimes do well at higher ratios.  So, it means use a cratio=0.33 or less for now.

At this moment, I am listening to a major improvement, but need to test more.  This was originally single band, so it has come a long way already :-).

John

Good news -- added a few fixes, and now sounds better at higher compression ratios.  It did really well as a single band, but there were a few nits as multi band.  It still has a bit too much response skew between channels, but now is MUCH MUCH more robust.

Sorry for the 'too early' release.  I do that sometimes.  This version 'C' is much nicer.  Still not perfect, it is meant for a devloper to grab it and add it to something useful!!!

Current criticism -- attack time a bit too fast for HF.  The decay time is much better.  MUCH MUCH less pumping, a correction in the front end filters.

John

2:1 compression is actually pretty severe, and it is still okay sounding.  It is 'fernando' from abba.  The uncompressed version is also  online:

abbaMB-1.5.mp3  -- Chiququita  at 1.5:1 with the multiband compressor
abbaMB-2.0.mp3 -- Chiquaquita at 2:1 with the multiband compressor
abba-1.0.mp3 -- Ciququita uncompressed.

Note that the compression done by the RMS style compressor is very smooth, and can be very fast and still listenable.  So, don't think that 2:1 isn't much compresison.  It is VERY FAST compression, with minimal distortion.  I screwed up the previous versions of the compressor (I am really good at making something work, then screw it up at the last minute.)  This one 'C' is fixed.

here is the repo location with the examples -- this is using the 'C' version:
https://spaces.hightail.com/space/tjUm4ywtDR

I added a Dionne Warwick example -- I also retuned the compressor to be more automatic (remember -- the compressor is a platform to start from.)   After the RMS detector algorithms and the input filters -- I'll probably have to explain how to adjust the design (it is pretty much all parameters in the class initialization for the RMS filters.)

I created a more dynamic attack/release scheme, where there is the inter syllable and intra syllable compression.  The intra-syllable compression is fast, and the inter syllable compression 'moderates' the behavior of the intra-syllable compression.  Basically it helps to mitigate pumping and the over compressed sound.  In the source code, there are several gain control filters, one is rfilt1 -- that is the inter syllable compression.  The intra-syllable compression is rfilt3 -- and probably doesn't need much tuning.

If someone is REALLY interested, I'd be willing to document the scheme -- but it does approach as good as it can be.

I didn't spend a lot of time on this specific example -- but I already know how to tune the attack/decay learning biases -- and they are definitely tuned JUST FOR THE GENERAL CASE.  The attack/decay control is much more capable than a normal/simple nonlinear capacitor scheme or a simple dual time constant scheme.  This is almost a learning scheme.

For some proof, listen to the 'dionneMB-2.0.mp3' example -- that is 2:1 compression, while the original is dionneMB-1.0.mp3. The amount of compression is excessive for the material, but I wanted to show how robust the compression scheme can be.  After these settings are chosen -- they don't need to be touched again no matter the kind of music.

There might be some desire to change some of compression characteristics, and the facility is in the code to change the compression timing/behavior and learning in almost any useful way.  The goal might be to have some fun with this thing!!!

Again -- look for the 'dionne' filenames, and 1.0 is no compression, and 2.0 is 2:1.

Repo: https://spaces.hightail.com/space/tjUm4ywtDR

John

The longer term behavior is improved on this version.  Also, the repository contains an example compressed with this version (dionneMB-2.0.mp3), previous versions have been supersceded.  It is even better now at 2:1 compression.

If you look at he progression of changes, you'll notice few changes -- the parameters and slight coding changes are all that is needed to make the significant improvements.

repo: https://spaces.hightail.com/space/tjUm4ywtDR

New compressor is attached also.

There will sometimes be problems when running the compressor.  I just got caught by it, and only just now noticed it.  There were .dll/shared lib problems -- and sometimes the compressor would 'run' but be very stupid.
This was an embarassing surprise, because even though I wasn't completely testing it on windows, I was verifying it, and it was working correctly on my environment!!!

This version has major cleaned up source code, and also will actually work on non-cygwin envrionements now!!!  Sorry about this trouble -- very frustrating.  This also screwed up my DolbyA compatible DHDA decoder distribution.  Somehow, some new libraries slipped in without me catching it.

REALLY APOLOGIZE.  The compressor really does sound nice...  REALLY DOES!!!
Everytime I try to do something nice for someone, I screw up terribly -- I can't do anything right sometimes.

John

There are several important switches, a new one has been added -- '--maxgain'.  It is already in the code.

So, the actual depth of compression is manipulated by the three switches (along with the signal itself):

Sorry for the confused nature of the switch values -- I really am not focusing on this very much -- just a distraction...

--thresh=xxx   it is the pivot level.  It is actually the negative of the number of dB where the gain pivot is.  Values
between approx 3dB (for -3dB) through 10dB (for -10dB) are probably most useful for normal level signals.

--cratio=xxx is a REPRESENTATION of the compression ratio.  The real compression ratio is 1.0 / (1 - cratiovalue).  So, a --cratio=0.50 means 2:1 compression, --cratio=0.33 is 1.5:1, --cratio=1.0 is inf:1.

--maxgain=xxx is the maximum gain produced by the compressor.  Previously, the default maximum gain was 9dB, now it is variable.

A useful mode to run the compressor is where --maxgain and --thresh have the same value.  It will act more like a soft limiter (very soft) than a full dynamic range compressor.

ONE MORE SUPER USEFUL SWITCH:  --info.
--info gives a running log of the gains & input/output  levels -- very useful!!!

Again, have fun -- and the --maxgain is VERY helpful.

John

I noticed that peak excursions are happening because of the slow 'RMS' style attack in the gain control.  So, I added a simple two stage peak limiter, and an additional switch: --maxlevel.   The default value for '--maxlevel' is '-0.5', so the default state is as if the user types '--maxlevel=-0.5' in the command line.   That is a good limiting level that avoids most peak excursions.

I didn't write the peak limiter to be infinitely fast (I didn't want to add an additional delay -- I guess I could have -- no biggie, it is done A LOT in the DolbyA compatible decoder.)  Anyway, the fastest attack time of the 'limiter' is about 250usec, which should be fast enough for most cases, esp since there is a 0.5 dB slop room.  Note that the peak limiter is on a channel by channel basis instead of a total of both channels, so some programs like sox will still complain, but by far MOST of the time, there won't be an excursion much beyond the highest representable level.  I also normally use --outgain=-3 just in case (which keep clipping from happening very often also.)

The peak limiter is NOT meant to sound pretty and can uglify the sound very quickly.  SO, I suggest using it only for the peak excursions.  I did not bandwidth limit the output nor did I shape the attack curve -- so it is a fairly raw limiter.  Not as bad as a clipper, and that is the ONLY positive thing that I can say about the built-in limiter.  To disable the limiter entirely, just set the '--maxlevel' to a high level, like +10dB.

Here is the next version of the 'compressor'.  I'll probably be laying off of it now, but I'll be glad to look-at and fix bugs as they come around.

John

After running lots of tests -- I still didn't like the number of peaks in the output signal.  I did something fairly aggressive and added in some faster 'RMS-style' compression at lower compression ratio to bring the peaks a little better under control.

This 'better' peak control along with the limiter should produce 'safer' output.   The result does sound a little more compressed, but the output signal is also more useful.

You can have both versions (version F and versionG), and have the source code for both (do a diff between the different versions of the file 'audioproc.cpp'.)  Which is better for you?  More peaks -- maybe a LITTLE more natural sounding, vs better peak control?

I might someday add a swtich to allow some customization of the result.  Both versions of the compressor sound 'okay', but it is all a matter of preference and taste anyway.

I am attaching the new version of the compressor (of coruse, with source also.)
Hopefully, this might be a learning exercise for some -- others might find my attack/decay classes (I only included a few of them) to be very useful at times.

John

What you use to split audio into several bands?

What you use to split audio into several bands?

I used a general purpose IIR filter generator subroutine -- produces the z transform for a 2nd order 's' domain filter specification.   The subroutine builds the IIR filter coefficients based upon the sample rate, and the filter function f(s)=(a0*s^2 + a1 *s + a2)/(b0 * s^2 + b1 * s + b2) is the filter specification.  Most of my filters are HPF, and since it doesn't need anything special, I do a group of subtractions with several filters to get the bands.  The source code for the IIR filter code itself starts at line 803 in agcfilters.h.  The magic subroutine that does the bilinear transform is called calcbilinear2 (actually a function template), resides earlier in the agcfilters.h file, and it does most of the hard work.  The filter spec is essentially the s transform stuff plus the typical hint for the 'match' frequency between analog and digital filter characteristics.  I use a varation of this scheme for my DolbyA decoder input filters (but the input filters are more dynamic and are recalculated on the fly.)
The filter scheme works great, and I suspect that the bands could be optimized for minimal artifacts.  I don't believe in the 80Hz base frequency for the LF on DolbyA (for a compressor), so I chose 160Hz.  Frankly, 240Hz might be a little better.

(Some hints on creating an HPF/LPF filter):
For an HPF filter, the specification looks like this:  f(z) = s^2/(s^2 + s * 2*pi*fc/q + (2*pi*fc)^2);.  The various parameters are obvious -- pi is 3.14159265..., fc is the HPF cutoff, q is the filter q.
For an LPF filter, the specification looks like this:  f(z) = (2*pi*fc)^2/(s^2 + s * 2*pi*fc/q + (2*pi*fc)^2);

I might have remembered these incorrectly -- so look them up under 'second order filter', and a wiki lookup is likly much more reliable than my memory :-).

I was VERY careful to make sure that the HF band fell fairly deep into the middle frequency range (2.6kHz) so that the levels for each band will be better balanced.  Don't want to use too high a frequency cutoff for the highest frequency band -- it needs some MF energy to make sure that it doesn't gain-up too often.

If you have any more questions -- just let me know.  I do have further improved versions of the compressor that have slightly better peak level vs. RMS behavior balance.

I can certainly help with simple DSP matters -- I know a lot about certain things, but very little about others.  I have some pretty good references available online and I can send them to you legally -- might help with audio DSP matters.

John

I spent lots of the previous post on how to calculate the filters, but I didn't talk much about how to do the filtering.  I have a C++ class called dfIIRHPF (HPF in the name is actually legacy and misnomer, it will work for any kind of standard 2nd order filter -- I just cobbled it from my DHDA code, and stripped out the proprietary secret sauce that is part of the magic in the DHDA code.)

So, an example of using the whole mix of filters is based on this idea:  A filter interpreter class (which does the filtering at very high speed), and the filter generator subroutine, the filter specification array, and then the owner of the filters (where the classes are instantiated.)

The filter interpreter is overly complicated for what is needed -- but you can grab that and convert it to any language.  But in C++, declares the filter interpreter (which calls the filter calculator subroutine 'calcbilinear2' when needed), and when it is instantiated as a class, the arguement is the filter specification array.  There are some examples (for exactly the compressor) close to the line 802 area of agcfilters.h.   I didn't just use magic numbers in the filter specs, so you can see pretty easily how to tune them as you desire.

John

Fixed some serious bugs, limits peaks much, much better. Compression a little looser, but the limiting is more absolute (and fewer errors.)

Will still sometimes go overlimit, but not often.  I decided not to add a delay yet, but will probably do so for the next release.

John

This version has better text RT display along with much better sound, along with much better peak limiting.
It now has a real version number -- I am comfortable with the general quality now.
You still get the source for future hacking on the code, etc.
For the nice RT text display, add the --info=10 switch -- it also gives peak limiter information -- example:

S:         16, INPUT(RMS:-22.83dB, -3.65dB), OUTPUT(RMS:-18.65dB, -1.21dB):
    CP LL( -2.07/ +2.58/ +4.90), M( +0.85/ +3.30/ +4.18), H( +1.35/ +3.87/ +4.78), PK( -1.95/ -0.00/ -0.00)
    CP RL( -2.23/ +2.58/ +4.90), M( -0.91/ +3.30/ +4.18), H( -0.41/ +3.86/ +4.78), PK( -3.70/ -0.01/ -0.00)

Here is the (long) BASH command line that also uses my DHDA DolbyA compatible decoder as input (this happens to be ABBA SuperTrouper DolbyA compatible decode, and then compression):

sox -v 0.50 "/music/agflac/raw1/ABBA*06*" --type=wav --encoding=floating-point - rate -v 96k | ~jdyson/ap/damulti/da-avx  --affinity  --ingain=6 --outgain=0.0 --tone=-22.67 --allow=no --info=0 --xfinalize --clarify| ~jdyson/ap/comp/comp-win --cratio=0.67 --thresh=3 --maxgain=6 --outgain=0.0 --info=10 | play -q -

Even though the compressor R+L channels have same gain, the peak limiter is per channel, and the limiter
information is applied to the total gain numbers.
First is the input/output levels.
Next is the Left channel info (LF, MF, HF, and then PEAK limiter) gains.
Next is the Right channel info (LF, MF, HF and then PEAK limiter) gains.
Each triplet for each band is (mingain, avgain, maxgain) over the 1second sample interval.

For sound quality -- the compressor is much better balanced, and has better short term/long term stats.
The peak limiters have been corrected, and is now three stage.  The goal is to first do a normal gain control
limiter, then subsequently faster and faster gain control that ends up being soft clipping at the end.   The peak limiters are essentially inaudible in most cases.

Remember the supported switches:
--cratio=xx, where the real compression ratio is CR=1/(1-cratio).  I know -- I should have fixed this.  So, for a true CR=2:1, then the cratio needs to be 0.5.  For a true CR=inf:1, then the cratio needs to be 1.0.  For a true CR=1.5:1, then the cratio needs to be 0.33.
--thresh=xx, the pivot point for the gain -- but negate the dB (I should have fixed this also.)  A typically good number is 6 (means -6dB), or 9 (means -9dB)
--maxgain=xx, the maximum gain-up for the compressor.
--maxlevel=xx, then maximum output level -- is the setting for the peak limiter.
--ingain=xx, the input gain
--outgain=xx, the output gain.
I have no external control for the attack/decay speeds or for the band frequencies.  If you want to play with those -- you have the source :-).

If you have any usage questions or want to discuss the source code -- just contact me through this group...
John

Just wanted to warn -- the limiter is still a little too intrusive.  The limiter certainly helps, but sometimes will become obvious well before I think that it should.

There is a new version coming soon, with a better limiter (design is mostly the same, minor changes and new parameters.)  The new prototype is more hidden from hearing.

The compressor itself is pretty good as-is.  If I have time, I might make the compressor slightly tunable, but it is already set-up to be close-to-optimal for hiding itself.   I'll try to make an option that gives different sound qualities.  Don't expect a new compressor before 8 Oct2018 (2 days.)    If you want to experiment with the compressor this weekend, the most recent download will be your best bet.

For compiling the compressor, clang++/llvm will be your best bet, and the source is optimized to get the best performance when using clang++/llvm.  I haven't compiled with g++ in a month or so -- there might be some changes needed again in the vector code -- but shouldn't be very bad.  Originally, the C++ headers/libraries used on the compressor were written to work best using g++, but I found that clang++/llvm is a little smarter in manipulating certain kinds of vectors, so that is why I am using clang++/llvm.   At one time, I did use the Intel C++ to compile with the C++ headers, but for my specific situation, ICC++ was no better than any other compiler.

If you have questions or problems using or compiling the compressor, just let me know -- and I will really try to help.

John

Oh my -- at least I distributed source.  There was a serious bug in the compressor where the freq response was messed up.  It was a stupid phasing error, but the problem is fixed.

There are a couple of bugfixes, including brighter sound with more correct high and low end.  Also, there are minor fixes for the limiter phase -- it is a bit more silent when activated.  (I have only spent perhaps 4-5Hrs on this code since it was written, so it is NOT perfect.)

However, this version is MUCH MUCH more useful -- sorry about the 'blank' that I shot off last month.  I hope that most people think of the compressor as a 'stocking stuffer' for Christmas.

Source is included, but the executable without excess baggage resides in the compdist directory when unpacked.  The 'comp' directory contains all of the source code and makefile.  Meant to be built under cygwin with the clang++ compiler.  Should be mostly portable for many POSIX style systems.  Native windows (not the simple command line as I have distributed) would take more work, but a command line port even to GCC shouldn't be a lot of trouble.

AS USUAL -- PLEASE feel free to post criticism or thank you.  Either one (or none) is welcome.  I'll fix bugs as soon as I am made aware.

Attached is a zipfile containing binary and source code.  It is yours to use -- please give me a bit of credit in documentation if you use the code or redistribute commercially.

I have a listening repo -- and there are two examples with the RMS compressor (almost exactly what was posted a few weeks ago -- probably NO changes) in action.  The compression was mild, but it really makes no difference using higher settings, because the compressor will do only a limited amount of 'damage' anyway.  The purpose was for demos, and the original material is online also. 

There are also a few examples using the DolbyA compatible decoder (lightyears more advanced than the previously posted versions.)  It REALLY DOES do better than a real DolbyA.  My examples have been EQed, so might sound a little fuller than commercial versions, but the HF is very close to 100% correct.   The dynamics on the examples re very good when considering the material.

The repository for listening is at:  https://spaces.hightail.com/space/yDG3L339Rn

Update -- I just reviewed my sourcecode and found an evil copyright/license in it.  Keep this posting as a reference for legal reasons.  You have full rights to do whatever you want with the compressor code.  I'll create a new version with a corrected copyright sometime today.  Sorry about the evil source-code terms -- they'll get fixed soon!!!

So, the 'external' agreement is that I am giving you permission to use the source or the binary any time.  Just give me credit (informally) in the documentation if you sell something based on this code.

John

Sorry that this took so long.  I intended to fix the compressor on the same day, but I became infinitely busy with the DolbyA compatible decoder, but finally -- I have tried to make the source/binary as legal as possible for you to use.  I am working with so darned many details of everything, that I let MANY mistakes pass through.

John

Thank you for this plugin.
I need some help, can you compile this software as foobar plugin or  standalone soft or vst for common user? I mean binary or exe file.
Thank you.

Thank you for this plugin.
I need some help, can you compile this software as foobar plugin or  standalone soft or vst for common user? I mean binary or exe file.
Thank you.

Unfortunately, it is a stand-alone program.  There is a makefile in the source tree -- but if you look above in previous messages, it is a command line program only.  It COULD be re-done to be a plugin, as most of the actual high level calls are in a short, perhaps 100line C++ section.  (The supporting routines are much more general purpose than needed.  So, a lot of code can be removed there also.)

I am willing to strip down the code to the minimum, but someone else would have to do the plugin interface.  (I am 200% occupied on the DolbyA stuff, soon to be DBX/C4/etc also.)

If you want to run the compressor in realtime, you can pipe its input from sox, and then pipe the output to sox or the sox-play command.  However, for the 'plugin' thing -- that is beyond the scope of what I can do without spending some time on learning the interfaces/etc.  Sometimes even, the plugin structures might not match the program structures (my programs all use a message passing scheme for multi-threaded operation.)  It might even be true that my larger programs (the compressor is very small -- it will most definitely fit) might not fit into the framework of some plugin schemes.  That is why I don't consider plugins at this time.  (My dolbyA decoder, running in full quality mode -- all kinds of distortion reduction/etc -- takes almost 100% of four cores on a Haswell to run real-time.)  If running in a mode that is as defective as the real DolbyA HW, it can actually run about 10X realtime.

(This is all the reason(s) why I don't do plugins right now.)
Again -- I am VERY willing to help someone if they wish.

John

Oh well -- if someone is REALLY bonafide interested in writing a plugin, I will happliy strip the program down to the minimum to make the porting easier.

John

Hello John!

May I ask you if your multiband compressor would allow me to pass the desired settings by using arguments?
I have tried Sox' mcompand. It should work, but it doesn't, so I'm in desperate search of a replacement, and your standalone multiband compressor looks just amazing too me.

Just in case you want to know what I would like to be able to do with the multiband compressor...
I have documented my efforts in a PDF file here:
https://drive.google.com/open?id=1UmUHAR6RAD7RfuqhKpgdOLqGkBzehb4l

But as I said, I think mcompand in Sox is broken, and I didn't get any reply in the Sox forums, so I'm looking for a replacement.

I took a quick look at what  you are trying to do -- my compressor doesn't do quite what you are trying to do.

First, it is a straight 1:1 through probably 100:1 (not sure, haven't checked recently) dB linear compressor, with RMS detection.  There are three bands (could be modified for up to 4 bands pretty easily (and the code actually supports 8 bands with a little bit of work) in source code), with an LF band (20-120Hz), MF (120-2600), and HF (2600 on up.)  The reason for choosing these bands are similar for the choices for DolbyA -- the LF band helps to mitigate modulation of the MF band by LF percussion.  If chopped up too much, there are effects that most people don't realize, so I have kept the number of bands relatively small -- and also kept the MF band for the entire vocal range.  It isn't cool to chop up the midrange -- bad things happen to the sound.  It is already problematic to chop up the audio into different bands -- the HF band can cause troubles with too deep compression.  (The compressor still works well, but the biases in the frequency response are an unintended consequence.)
I do have some algorithms that mitgate the frequency response skews, but haven't included them into the very simple compressor.

For attack/release -- it uses an algorithm that adapts to the material (very much so), much much much more adaptive than most compressors.  It maximizes the compression with minimal audible artifacts given the specified compression ratio and maximum gain.  It is meant to be 'knob free'.

The variable 'knee' per band isnt implemented that way -- it is meant to be a simple compressor -- all bands have the same compression ratio and threshold.  Below the SINGLE threshold for all bands, the compressor increases the level.  Above the single threshold, the compressor decreases the signal level.

Above a maximum level below 0dB (the digital clipping level), one might specify -1.0 dB relative to clipping, there is a simple (but finessed) limiter with inf:1 compression ratio and adaptive release behavior also.  Upt do about 2-3dB of limiting, it is essentially inaudible, but it doesn't clip or do nonlinear things to the signal.   It is meant to be a last chance signal limiter to avoid digital clipping, not for a creative effect.

So, that is the general idea of the compressor.  I don't have time (right now) to change it to 4 bands or set up different thresholds for each band (each upgrade item isn't really difficult at all -- but the compressor design is meant more as an example, or to use as-is.)

Some day, in the future, if there is interest, I might spend several days to upgrade it to be more flexible -- the compressor as-is is the result of perhaps 3-4Hrs work at most (I cobbled together a bunch of libraries/C++ classes/etc from other projects.)  Even the RMS detector is meant to be used in my DBX I/II decoder project (upcoming.)  I have developed a lot of technology for future use!!!  The compressor is a small program implemented using a VERY small set of libraries in my collection of tools.

If you think that given the description (and looking at the usage notes in previous postings), the compressor is close to what you need -- I can give you some help in using it -- maybe changing the bands/etc.   I might have time to add a 4th band, maybe even make the bands variable (based upon a command line argument), but I am 100% occupied for the next week or so at least..  I am finalizing the best ever DolbyA decoder -- and it is a VERY difficult project, fully occupying my ability to think...

John

Hello John,

I understand your description. Thank you very much for your time.
I would really love to use the knob-free approach, but unfortunately I can't.
Instead, I need to stay as close to the mentioned Multiband Compressor as possible in order to produce nearly the same results.
I would totally appreciate it if you could find the time for tweaking your multiband compressor after you're done with your DolbyA decoder!

Hello John,

I hope your dolby decoder comes along fine!

You haven't had the time to look into "my project" yet, have you?

Cheers!

Hello John,

I hope your dolby decoder comes along fine!

You haven't had the time to look into "my project" yet, have you?

Cheers!

I am totally overloaded on the DolbyA compatible decoder project. Hopefully, the DolbyA compatible decoder program is really starting to be able to satisfy the most critical users, so the pressure might be diminishing a little. might be slowing down in a few weeks.

I have recently been doing some serious rework, where the very advanced 'super RMS' detector (RMS measurement, but lower ripple) didn't emulate HW behavior adequately to reliably properly decode DolbyA encoded material.  I finally gave-up (on my detector which is really good for normal compressors/expanders) and did a precision implementation of the REAL HW detector and nonlinear characteristics (lots of diode equation type stuff), and am now getting robust results.  The emulation is so very precise now, that there is a different mode for the old 301A style DolbyA (the follow-on DolbyAs like the cat22 do act differently from the original 301A), and the results have been stellar (but this breakthrough just happened in the last two days.)

Previously, the decoder was good enough for listeners (really nice in some ways), but didn't emulate the decoding precisely enough for serious recording engineers.   The advanced features still persist (the anti-distortion stuff), but now the front-end emulation is so pristine (becuase the 'diodes' that I am using are perfect, and the design is very clean -- my capacitors in my software are very, very good :-)), that the anti-IMD operations are less critical for providing superior results.

This stuff is 'no-joke' when it comes to tedium and in-some-ways complexity (not really complex, but when starting from scratch, it is not easy.)

With this comment -- trying to explain why I have cannot allow myself to be distracted -- I am intellectually/mentally and emotionally exhausted, yet there is still significant work needed for the decoder (GUI interface for a seperate GUI program -- avoids license encumberances) , and some other things.

I like to do compressors/expanders (I actually prefer intelligent expanders), but I have made promises to deliver great results on the decoder), and it has only been extreme concentration that has helped make as much progress as I have on the project (and sometimes, I know that I haven't done good enough.)

I WILL take a look at making the compressor more flexible when I have time!!! I really want to look at it -- I have some really good ideas for exceptional quality (probably cannot divulge the more advanced stuff from the decoder project, but still have some potentially helpful technology for when I can work on it.)

john

jsdyson,
Do you have any updates on the project?

jsdyson,
Do you have any updates on the project?

Actually, yes I have made some upgrades, and some better control of the compression (maximum compression gain/loss -- so that it stops compressing above a certain gain.)  Also, I upgraded it to an intelligent 4 band -- it doesn't over-hype the high end when gains are high, but will compress them if they are too intense (there is a relative threshold setting, for the upper two bands also.)

Made some qualitative improvements -- better attack/release control, and the peak limiter has less distortion -- it can run more deeply with less intrusion in the sound.
The bad news -- the state of the compressor source code still sucks.  I have been working so crazily on the DHNRDS decoder (happens to do a good job of decoding material that is DolbyA encoded) that I haven't been able to focus elsewhere.  I plan to make something -- the new compressor -- available in the Sunday/Monday time frame (30thJul2019) hopefully.  The new decoder work is lightyears beyond my previous attempts (an actual audio professional jumped in to help.)

About the DA decoder:  I am attaching some snippets of some Olivia Newton John (truncated for obvious reasons) -- but the new DA decoder JUST MIGHT someday help to improve the quality of some of those old recordings as distributed through normal channels.  The DA decoder is a pretty raw professional tool, and inexpensive -- but works wonders (at least, in my opinon as it's daddy :-)). * If there is something sentimental, and you already have a copy, and I already have a DolbyA copy -- ill make a copy -- if REALLY desired.  We aren't selling the decoder to consumers (support issues, etc) -- but I am willing to do one-off decodes and even supply a timed-out version gratis if it is really needed by consumers only.  The pros need to go through my project partner and will get better support than I can give (he has a nice manual/etc.)

Here are the snippets:  https://www.dropbox.com/sh/l9vjbst22duw0ua/AACR2MGVhBeVfA286gUSEXnAa?dl=0

Give me another day to make the compressor available.  It is all written/debugged (well, you know what I mean.)  I'll package it up -- forgot, have to build a Windows version, so it will take time anyway.  I'll allocate time tomorrow to complete the build on Windows.

John

This new version of the compressor tends to sound better and the limiter works more cleanly.  However, the source code is just as ugly as before.  It is now 4 bands with good managment of gain differences between bands.  The compressor usually doesn't sound very obtrusive at all.

It is okay to use the compressor itself and the source code in any legal way -- if you happen to sell something derived directly from this code, just mention me as someone who helped.  I don't reserve any rights to the code -- use it at your whim -- just be a good citizen.

It compiles directly under the cygwin environment (actually written on Linux.)

Most trivial usage is very simple command line:

> comp-win --inp=infile.wav --outp=outfile.wav --cratio=2.0 --thresh=6.0
(Be careful using any new program like this -- there is nothing intentionally bad in it.)
(adding the --info=10 switch allows you to see the second by second progress of the processing.)

Also attached is a sketchy 1 page guide about the available command line switches and mentions various behavioral options.
There are NO attack/release adjustments, the compressor will generally adjust attack/release as effectively as a user can.
It is audio-RMS based detection, not EE-RMS style, so it is as inobtrusive as a simple compressor can be.

If you have questions, I'll try to answer them, but please read the sketchy guide first.  Also, make sure that the .dll files are in your path (or sitting next to the program binary.)  This will normally be true if you unpack the program into the 'comp2' directory as it normally would do.  A standard Windows install is NOT used, this is NOT a professional product,  doesn't even qualify as a good hobby quality product -- but it works, and might even show some interesting/useful programming techniques, even for a veteran C++ programmer (very SIMD centric.)  Even by my own hackery standards, the code is ugly -- sorry -- what could I do in a few hours.  This note and quick ref guide took longer to write than the cobbled together source code!!!

The program has only be tested on a feeble ATOM Windows laptop (other than the original Linux development), and appears to work.

I you want to use a simple compressor, and have FULL CONTROL over it -- you have the source code with this program, and might be a start to modify for different bands, etc.  The architecture IS limited to 4 bands without more than a few simple changes, but could easily be changed for fewer bands or change the frequencies.

Good luck!!! (you'll need it :-)).

Does your linkwitz-riley filters use allpass IIR to give flat frequency response when summing all bands together?

I simply use 2nd order filters that when summed together result in a flat response.  Of course, as the gains change, then there can be odd effects -- but really don't hurt -- the freq response is all over the place with multiband.  Since the bandpasses are soft (moderately low Q except the LF bad which purposefully has a high Q for the same reasons as DolbyA), and the bands are separated by subtraction, there isn't a really bad problem.  If you listen, the effects of the filters aren't all that obvious except for the expected bandpass effects.  One more thing -- the gain variations are restricted in a way that the phasing isn't a problem -- even if it was a problem.

I did consider using the multiple butterworth trick, and it IS a cool trick, but I didn't do it.  There was absolutely no reason for such a sharp cutoff, and the phasing issues aren't very severe.  (Listen to any proper DolbyA encoded/decoded process -- I used the same scheme.)*  The sharp cutoff of 4th order is more problematical, and only a few dB of separation between bands is needed anyway.  You can take the code in agcfilters.h, and implement two 2nd order with Q=0.707 for each band, and get the tight separation, but my experiments show no advantage, and the tight bandpass actually shows a disadantage except for the very low frequencies.  The lower Q around 0.500 (I did goose up the Q a little) is the same as a Bessel for 2nd order -- close to linear phase.  I didn't hear any bad effects up to the full single order Butterworth Q=0.707.

* If you listen to undecoded DolbyA -- probably 2/3 of the material that was recorded before 1990, but on digital -- giving that often described 'harsh' digital effect, that is UNDECODED DolbyA -- people never complain about the phasing, but just the boosted highs (I also notice the flattened stereo space.)  I can at least compare decoded and undecoded material because of the ability to decode the too common 'feral' DolbyA material.

Interestingly, on my DolbyA compatible decoder, I use linear phase filters for the high frequencies, where the phase mismatch between my linear phase and the HW standard low Q filters hasn't caused any notice or complaints, except with the better control of errant peaks by the linear phase version of the input filters (well, I did use the Q=1.170 for the 74Hz filter as in a true DolbyA because of the need to emulate the weird phase shifts there -- otherewise the gain vs. freq at LF because really weird BECUASE OF THAT HIGH Q.) 

Using the multiple butterworth scheme might keep errant peaks from happening, but with my comparison with DolbyA HW and my SW implementation -- I haven't seen that being a terrible problem -- just noticeable, with my scheme sometimes giving about 1dB extra peak headroom.

John

One more thing -- If I was to do the compressor over again, I would have used FIR linear phase filters throughout (my goals didn't include realtime play), but like in the DHNRDS DA decoder, the input/output files would be time synchronized (the play-time compressor doesn't do that -- it is a toy,  the DA decoder is NOT a toy.)   Also, I wouldn't have used the same kind of RMS detector, but a different technology based on Hilbert transforms, but yielding the same general results -- and NO RIPPLE.  When/if I ever did a professional calibre compressor -- I still have about 5 NR decoders to do first -- it would be different in a lot of ways, but still use the same SIMD techniques.  If I didn't use SIMD on the DA decoder, it would be much slower.
My own work -- when focusing on perfection, is REALLY crazy perfect (ask my project partner.)  You would not believe the modulation products from the fast gain control that I figured out how to cancel.  Most people in the field would claim that it is impossible (I'd expect that Orban hasn't figured this one out, well maybe just him/his company), but the modulation products are REALLY combed out really well on the DA.  I would do the same kinds of techniques on the compressor -- it would be faster, yet not produce the typical 'too-fast' attack/release distortions.
I haven't worked out the details on the conversion from the DA to a compressor, but I can give some hints if interested.  There WILL be some papers on it in the future.  (The DA decoder produces infinitely more clean results than DolbyA HW does because of the advanced DSP techniques -- especially obvious on vocal chorus -- individual voices are maintained instead of a blob like so common on material like ABBA!!!)

If you are interested in the advanced techniques not used by the toy compressor, I am certainly willing to give hints.  Most of them use Hilbert transforms and in some cases, a kind of iterative technique.  I don't think that a normal compressor would need the heroic techniques used to improve DolbyA emulation behavior (really fast release times in the 30msec range -- produce lots of modulation products that don't really sound *bad*, but lots of detail is lost.)

Why you use hilbert transform at all? Isn't that just giving you 90deg phase shift? Do you use FIR or IIR filters for that? Do you use decorrelators?

I have some major improvements for the multiband compressor -- it really was meant as a toy.
ADD-ON:  the poor-mans mastering code in the DHNRDS provide a much better 1 band, 2 band 3 band compressor, and also a limiter and anti-sibilance capability.  In a year or so, I'll release the source, but the code is available now (I'll annouce it elsewhere on the forum.)  The compressors DO use the Hilbert scheme for DA, in fact use the DA detector method, so does the limiter.  Unfortunately, the DA detector scheme isn't super fast, so is not a hard limiter, but is indeed very effective to keep normal peaks under control (but not perfectly.)   The DHNRDS actually supports 3 compressors of each time, with user specified compression ratios and thresholds -- so a varying curve can be implemented.   The DA detector is almost perfect given it is a non-RMS scheme.

However, on the DHNRDS there is a great need for precision.   This means that phase errors are a non starter, and typical IIR type phase shifters (which are essentially a Hilbert transform) aren't very precise.   In fact, the is a great need for precision, both in the anti-modulation distortion code along with the detector.   Also, Hann windows need not apply -- we are talking about a need for precision so great that the errors associated with the window function are important.  The DHNRDS emulates a piece of HW, and there is ZERO (absolutely ZERO) room for errors away from the needs for emulation.   If I was doing a GP compressor/expander, then it might be okay to allow SOME ripple in the transform, but I'd prefer just to do a GOOD FIR scheme along with my carefully chosen windows (when ripple might have otherwise happened, the ideal window mitigates the resultant effects.)   Considering that a typical run of the DHNRDS for cleaning up consumer recordings requires approx 24 Hilbert adjunct detectors with the need to avoid modulation products down to approx 50Hz or lower, I certainly would prefer to use IIR scheme if no quality would be lost!!! 

There are two places where the 90deg phase shift is needed in the DHNRDS:

The anti-IMD code, which is an adjunct to the calculation for the DolbyA compatible attack/release.   The DolbyA compatible attack/release is a highly controlled waveform, is a two layer attack/release, where the second layer has a log attack and R-C release.   The first layer is a log attack and log release.  This emulation is on the nonlinear part of the diode, not the rectification -- making the curve very complex.   The Hilbert detection melds with the DA compatible calculation to minimize the gain control modulation products produced because of internal nonlinearites, the super fast attack/release times for a compressor/expander, and the resulting modulation of the signal created by intermod in the audio used to calculate the attack/release.   The Hilbert adjunct stops the resultent modulation of the gain control signal.   The purpose is akin to, but in addition to the ripple reduction associated with such detectors.   Part of the challenge is that the DA detectors are not true peak detectors, but a standard Hilbert detector does tend to be peak detecting.  The Hilbert detector design has to be modified to work in conjunction to the DA scheme. (DA needs 2-40msec attack and 40-80msec release, precisely calcuated to match the old HW.)  R Dolby was a genius, and designed the detector to go as far as possible to minimize distortion given old-time HW -- but the DHNRDS does MUCH MUCH better.

Note:  an important part of the anti-IMD design is that frequencies below about 750Hz must maintain exactly the same gain control shape as the encoding signal.  Nonlinearities in the encoded signal must be cancelled.   On the other hand, frequencies above 1-2kHz have the tendancy to produce modulation distortion, which has the tendancy to soften the sound given the DA configuration.   The frequencies between 500Hz and 1kHz can be the results of intermodulating of the rest of the audio band -- therefore causing a wierd kind of ripple in the gain control.   It appears that the two level attack/release is the root cause for this intermodulation -- doesn't seem to appear on conventional attack/releas gain control, which mostly just has conventional ripple which can be filtered.   The anti-MD below on frequencies below 750Hz is a bad thing, because it effectively changes the shape of the gain control, and actually INCREASES resultant distortion.

The anti-MD (different than the anti-IMD) code does some things that are normally deemed impractical -- it actually implements the gain * signal calculation to effectively minimze the creation of modulation distortion.   It does something similar to a highly finessed shaping of the gain control signal, but also considers the audio signal during the calculation. The anti-MD  doesn't directly affect the gain control nor the audio signal, but instead changes how the result of gain control is calculated.  The Orban patent tries to do something similar, except it moves the sidebands to the higher side (sounds smoother), while the DHNRDS minimizes the sidebands -- keeps them at an even level when gain is changing.

John

Then how many FIR hilbert filter have taps in your usecase? Note that there is no way to avoid attenuating low frequency magnitude content when using FIR filters.

Then how many FIR hilbert filter have taps in your usecase? Note that there is no way to avoid attenuating low frequency magnitude content when using FIR filters.

Well, IIR filters are worse -- they wobble worse.
For 'detection', the Hilbert detectors are purposefully non-working below 600Hz.   The reason for them in the detectors is to keep the multi-layer nonlinear detector compatible with DolbyA from generating ripple from IMD.   The detector enhancements REALLY make an important improvement.   Anti-ripple CAN NOT be used below 600Hz because the decoder must undo the distortion (hand in glove) created by DolbyA encoding.   Therefore, using anti-ripple techniques below about 600Hz  make the sound WORSE.

For anti-MD, it is much more complex -- but the Hilbert transformers are used in a nonlinear scheme as a part of the gain control mechanism to instantaneously control the slew of the gain*signal where the peak strength of the sidebands are suppressed.   This is a huge problem with DA decoding because of the continual, repeated super fast attack/release times.   This continous/fast attack/release creates a sideband fog -- which hearing perceives as a 'warmth', but when comparing with the original, it becomes 'imprecision.'   Slow compressors/expanders really don't need the anti-MD.   This goes WAY WAY beyond the Orban upper sideband stuffing technique.
What kind of Hilbert transformers are needed:   VERY SPECIAL.   Hann windows need not apply.   Since the process is nonlinear, the  transformers need to function down to about 30-50Hz, up to Nyquist -- can you say LONNNNG FIR filters?   I thought you could :-).  The filters must also be high precision to be effective, so I am stuck with DP math.   Wobbles need not apply, or if you do have wobbles, use VERY SPECIAL ones, which I do.   The windowing function is very carefully chosen to maximize sideband suppression.
For initial guidance I reviewed an important paper (I can look it up for you, if you want, and studying/understanding the characteristics, a certain family of Windows appeared to be optimal, which they were.)

I haven't seen my method published anywhere, but it just became practical with serious SIMD machines and lots of threads.  It takes a serious amount of processing to do correctly.  Once the technique is divulged, maybe some college student will be able to program it onto an 8051, but I just don't know how to simplify the method.

The results -- the FeralA decoder, has been posted elsewhere on this forum.   Look at the video for how much gain control is happening, with NO perceived distortion.   Some of those 10/15dB gain changes are happening at least 5-6 times per second, and there are 8 of them going on at the same time on the same signal.   (2-40msec attack time and 40-80msec release times -- even faster slew for 9kHz and above.)

John

BTW -- the previous compressor design is cr*p compared to the one that I am using in the FeralA decoder (it has poor-person mastering capabilities built-in.)   I think that a true RMS compressor with a Hilbert adjunct would be MUCH better.   The compressors in the DA decoder do use Hilbert adjunct to a DolbyA style detector.   I think that RMS would be better for GP applications.

Give me a few months, and there might be a  REALLY, REALLY good RMS compressor source available.   I have to better control the FeralA project first.

John

Well, to may understanding any band splitting to work correctly need allpass component. Otherwise it will now work correctly.
Also how bands are split are also very important, for example what order of filters are used.

Well, to may understanding any band splitting to work correctly need allpass component. Otherwise it will now work correctly.
Also how bands are split are also very important, for example what order of filters are used.

It is important to use filters with wide skirts -- NR systems are the most stringent, and they use wide filters to minimize modulation distortions.   Both DolbyA and C4 use wide filters.    Telcom C4 does something cool to get the benefit of wide filters and narrow band selection -- they  use different EQ for each, but generally the same bands.

* for small applications where there isn'tmuch gain change, the filter choice isn't as complex.

You can get by with more narrow skirts on compressors/expanders that don't need to reverse the process -- the modulation distortion doesn't have such severe consequences.
You don't really need allpass when you use either wide-skirts or use linear phase filtering (which adding all pass helps to TRY to create for analog filters.)   Doing it in digital CORRECTLY is much easier, but is frought with troubles for other reasons (e.g. the impossibility of doing a pure audio feedback loop like normally required by DolbyA for precise emulation.)   I had to calculate all of the effects of the feedback loop and synthesize them.

I haven't done much IIR splitting except for simple apps, because it (phase) is too sloppy - allpass isn't really an answer in digital.   Just design linear phase equivalents.   That is what I did for DolbyA where it has a Q of 1.17 for the 80Hz bandpass, but Q of 0.470 for 3kHz and 0.420 for 9kHz.   The phase twisting all over the place for the 80Hz bandpass causes a myriad of troubles, but I did resolve them, and that was the only IIR bandpass that I used.   I understand why he did it (Q=1.17), but the 80Hz (actually 74Hz) bandpass doesn't really mean 80Hz, because of the twisting all over the place.   The filter really ends up causing the lower 80Hz band to include up to about 160Hz.   This really screws up the MF (80Hz to 3kHz), which is really (74Hz to 2888Hz) band, because it makes my Hilbert detectors really tricky to avoid the problem of 'hand in glove' recovering the actual distortion caused by the fast 4ms-80ms attack and 80-160ms release on the LF/MF bands.  It is important to allow ripples of certain frequency to occur, otherwise the previously created distortion isn't undone.

For the 3kHz and 9kHz, I just created the linear phase equivalents of Q=0.470 and 0.420 -- works perfectly.

I know that I am talking about DolbyA (and certain initial work on Telcom C4), but the needs are similar to a compressor/expander, but infinitely more stringent.   The reason why it is more stringent is that the digital emulation must be precise.   For example, emulating the selected diode nonlinear characteristics in DolbyA required accuracy of the nonlinear parameters at the 1% level or less.   When designing a normal, ad-hoc compressor, the parameters are more up-for-grabs, and tunable for the application.   Not so for emulating a tricky piece of hardware -- stuff must be incredibly precise.
The DolbyA was designed by circuit -- that means that the specification is the circuit itself.  For telcom C4, they appear to have a design spec as it is easily defined (I have such a definition in my records.)
On the other thand, DolbySR is much more complex, and I am probably one of the very few that does have the circuit.   It is hell, and only in the last year figured how to emulate it without violating patents.   That, in fact, was one of my challenges in DolbyA.  It had to be a novel design (but allowed many more degrees of freedom, e.g. adding anti-MD, which is impossible in a normal feedback design.)
This stuff can be deceptively complex.
Take a look at the video in the FA decoder topic -- it sometimes does 60dB of gain control multiple times per second -- the gain control is crazy to emulate the layered compression used on (against) consumer available recordings -- it is nasty.

John