Skip to main content


If you are using a Hotmail or Outlook email address, please change it now, as Microsoft is rejecting all email from our service outright.
Topic: Have a working 'expander' based on DolbyA (not same design) -- works well. (Read 24730 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #50
Claification about the 'pseudo-DolbyA' decoder.  I want to make it clear (because this question came up in another forum), that this decoder often has positive effect on NON-DOLBYA-ENCODED material. In fact, the Rainy-days-and-mondays example that I gave the spectrogram for was NOT advertised to be DolbyA encoded.  The 'decoder' is an expander that undoes some analog sidechain compression.  Because of that architecture, and (helpfully) the use of a non-traditional attack/decay calculation -- it can be effective in other cases where the material was compressed with a sidechain (or even non-sidechain) compressor.   There'll still be SNR and some limited transient improvement, even if not DolbyA encoded (but is compressed.)

So, when I suggest that people give it a try -- it is NOT necessary to have 'DolbyA' encoded material, even though results from such experiments would be interesting.   I do have some material that is DolbyA encoded, but I cannot give it out as examples (2Tk mixed down master of pop music.)  Because of the kind of intentionally distorted dynamics in the original material, it is impossible for me to accurately determine the quality of the DolbyA decoding.  (every CD that I have heard based upon the material that I have sounds so very different, because the mastering engineers have really played with the 2tk master.)

Regarding the spectrograms:  The reduction of 'blue fog' (mostly hiss) occurred on compressed material that benefited from mild expansion (but not so much expansion as to badly distort the dynamics.)

The 'larger scale' expander can make more significant impact on dynamics, and it is a bit more tricky to do that correctly without causing unpleasant expansion artifacts.  However, the design of the 'large scale' expander (dynamics restoration software) is such that it is easy to set the parameters so that the results can sound better than the original.

John Dyson

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #51
I have posted a much more substantial demo of the available noise reduction using the 'four band expander' or psuedo-DolbyA decoder. This demo should work for most people who still have at least some hearing (remember -- I do have age related hearing loss, so sometimes HF NR is hard for me to discern.)
On my repo site, I have a few files which should pretty clearly show the expected NR associated with a Dolby-like NR scheme. This demo shows both HF and LF NR (which DolbyB only has HF NR.) The resulting NR is modest, but useful. The NR doesn't generally have a negative impact on the sound when it is needed.

I simulated the noise from a typical cassette deck -- so it is pretty obvious. Also, for a 'bonus', I simulated a little bit of turntable rumble that had been cut off at 20Hz. II have included the noise generation/addition script for additional review.

There is a definitely noticeable NR -- both files have been roughly equalized in volume. Due to the effect of of the expander, certain portions of the music show a difference in HF, but the difference (IMO) seems to make the music more accurate. The original piece seemed to either have a little too much HF emphasis or might have been DolbyA encoded.

Files at the repo site -- available for download and comparison:
carp-noise.mp3 -- noisy version of the music
carp-noisered.mp3 -- less noisy version of the music
carp-noise-stats.txt -- sox stats showing approx signal levels for the noisy file
carp-noisered-stats.txt -- sox stats showing approx signal levels for the noise reduced file -- shell script with sox commands used to add noise

The repo site:

This NR does work with files that are NOT explicitly DolbyA encoded. I just reran a test with a song from this 'shawn' person (youtube download), and this 4 band expander did improve the dynamics (tastefully sharpened the sound -- slghtly backed off some of the over compression.)

It order to avoid confusion, I am not explicitly talking about the more aggressive expander project at this time. I am really trying to help the community by trying to help improve the sound quality of (mostly) legacy recordings. It is possible to do more processing, but I believe that the processing should make the music/sound 'better/cleaned-up', not 'more' or 'overdone'.  I am trying to keep the results at 'tasteful' levels.

John Dyson

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #52
following up on the previous post -- I am attaching a spectograph output resulting from the NR associated with the various expanders.  I am concentrating on the 4 band right now, but this posting shows the results of the following:
carp-noise-8band-4band -- the spectrum for using both the 'pseudo-DolbyA' and full-out expander.
carp-noise-8band -- the spectrum for using just the full-out expander
carp-noisered -- the spectrum for using just the 4 band pseudo-DolbyA expander.
carp-noise -- the spectrum of the noise added signal alone with no processing.

If you look at the spectrograms carefully, you can see that the carp-noisered does help with the HF noise (it is not easy to see the LF NR also, but it is noticeable when listening.

carp-noise-8band (which I am not concentrating on right now, but the results are here) shows that it is a little better at finding the 'blank spots' to do noise reduction, but isn't quite as good at the ultimate NR at the highest frequencies.  You can see relatively blank sections in the upper-middle range where the 8 band found some of the noise to remove (it is both faster and has narrow bands to work with than the 4 band.)

When using both NR/expander systems, the advantages of both can be seen.   There are much larger 'lighter' areas where noise has been removed.  However, the darker areas with real signal are still there.

The 'carp-noise' and 'carp-noisered' examples reside on the repository that I have referred to before.  I'll send the results including the 8 band results right away (you should probably be able to see them on the repo by the time you get done reading this.)


Very major improvement to 4band expander (pseudo-DolbyA decoder.)

Reply #53
Got a major improvement -- source code and Linux binaries only at this point.  All along, there was a problem that sounded somewhat like a garbling that wasn't quite right (under certain very dense music conditions.)   This manifested on a 2trk master tape that I use for some tests, but has gotten better and better over time as I had improved the dynamics of the gain control (gain change vs. time.)

After I did a more careful code review, I noticed that there were some gain control operations that were happening against an overly wide band representation of certain parts of the audio signal.   This basically created excess intermod products.   So, I did a very careful review of the cases where the gain change operation was being done at slightly the wrong time and corrected that problem.  Also, there was a minor 1msec delay problem between the signal level sense and the actual signal itself.   This also caused a slightly rough sound.   These problems were all forms of intermod -- the bane of any kind of audio gain control.

All in all, these improvements are worthy of an early re-release of the source code.   On simple kinds of music (e.g. Brasil'66), the improvement is not really noticeable.  But some kinds of very dense music (e.g. ABBA stuff), the improvement is quite important.

This early release has some test code in it (commented out), and is a little more 'unclean' than usual.  However, the fixes are installed.   I'll release a windows version when I am done with the full release (windows is a bit of a pain in the b*tt for me to deal with -- I usually just use windows for circuit analysis.)

Attached is the new code.   It also resides on the repository.

John Dyson

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #54
After the above improvement to the psuedo-DolbyA, and some additional gain-control/signal synchronization on both the psuedo-DolbyA and the restoration processor -- I have gotten some amazing results.  It has been critical to shape the attack times very carefully to avoid intermodulation type effects (a kind of garbling.)   For ANYONE who likes ABBA (and a Carpenters example), give the below a listen.  I plan to release a semi-final version of the psuedo-DolbyA within days, and start considering/pruning down a release of the restoration processor.

Listen to these results -- they sound like ABBA -- but not quite, maybe like ABBA was recorded in the 2000's instead of the 1970s'.

Imagine VERY SMOOTH/CLEAR ABBA with a LOT less compression.

John Dyson

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #55
Attached is the latest pseud-DolbyA with very significant bugs fixed again.  These aren't the kind of bug where there is an 'off by one', but rather the attack times are more carefully calculated.  The resulting improvement is seldom heard except on HF intensive material like ABBA.   The included fixes (and the associated fixes in the 'restoration' processor) were critical for the amazing results that I recently got on 'cleaning up' some old ABBA stuff.   Up until now, ABBA always had a garbling effect when multiple voices were in play -- and the result was kind of ugly.   The problem seldom existed anywhere else, and most of my previous 'fixes' did  help, but were either work-arounds or fixes in minor portions of the code path.  These new improvements change how the attack (gain attack) occurs, rather than just being an instantaneous response or a RC delayed response -- there is a hybrid technique that I saw R Dolby use, and I started applying it in other cases -- with rather amazing results (the guy was a genius.)  My own implementation is software based version of the hardware concept.   Attached is the latest source code and build for the psuedo-DolbyA on Linux.  Like I mentioned before, the Windows version will be forthcoming when I have a chance to do development on my Windows64 box.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #56
Real nasty gain control bugs in my projects (partially due to hearing troubles -- fixed.):

I have been working on some hellish bugfixes -- I had lost part of my hearing recently, and luckily it had come back.  I made some serious mistakes in both the pseudo-DolbyA and restoration processor.  During the process of fixing the bugs, I have learned some semi-interesting facts about these kinds of high speed expander/compressors in the digital realm.  Note that  I have definitely fixed a lot of potential design issues over the years (incl mitigation of some pretty evil intermodulation distortions, etc) on compressor designs.  Well, my hearing lost approx 6dB at 10kHz, and I missed a huge number of artifacts in this new expander stuff.   The technical explanation of the bugs is basically -- Be VERY careful about attack characteristics.   After listening to some of my processors with my new hearing, I noticed very severe HF harshness.  Almost 100% of the harshness came from the idea that a gentle attack (I mean, gentle in a fast way -- e.g. 1msec attack time) is a good thing in the digital realm -- not necessarily true.   I had been softening the attack to make the middle frequencies work better -- but unknowingly at the cost of some HF hash.

SO -- when doing a review of the simpler software of the two projects -- the psuedo-DolbyA, the key 'notch' in the attack characteristics that come from the original DolbyA HW design does work pretty well also to help with the 'hash.'  A raw attack is not a good thing for a compander because tape machines do not maintain great phase character at high frequencies, so a bit of attack time roll-off is important.
However, even though this Dolby-notch does help the hash, it really needs more help in the digital domain where the signal frequency might only be sampled at a couple of times.   The indeterminate  timing seems to cause a bit of 'hash' when mixed with the gain control (THIS DOES NOT apply to many compressor designs that do the simple approach of SLOW attack/release times, like 20msec or slower.)   However, there is a critical design rule #1 that I didn't follow 100%, but should have (and this is most critical for fast attack times):  put all of your attack time constant on the first measurement of the signal level.  If one doesn't do that (or use another bit of technology that I had temporarily sidelined until now), then there will be an obnoxious raspyness if the attack/release times are quick (that is, when you might have a 1msec measurement, then an additional 1msec timeconstant somewhere, then a 100msec release time.)  That is a formula for hellish sound (esp on groups with lots of potential HF intermod like ABBA.)
So, when I do need to further filter something like:  1msec attack time constant, then have an eventual set of release times with a fast component (like a 100msec component for the release time), then either I make sure that there are no other time constants in the chain, or use a special dynamic filter which basically dynamically follows the signal -- and I'll release the code in the future which DOES significantly mitigate the problem of subsequent attack times -- that is multiple in series is troublesome without great care.)

The other problem that I ran into (which I knew about, but couldn't hear -- so didn't believe it to be severe.):  when one uses a medium attack time like 1msec, with a combined absolute value function (basically a linear rectifier), then there is still a matter of intermodulation (which is VERY bad with groups like ABBA -- multiple female voices with high pitch, intermodulating quite a bit.)  This nonlinear activity along with the 1msec filter is not slow enough to filter out the intermod, and is not fast enough to confine the instantaneous change in gain to a very narrow timeframe.  This narrow time frame is minimally inaudible.  I have previously had experience MOSTLY with square law detection, which seems to be a little less sensitive to the intermod than the hard absolute value function.  SO, it is critical to have the ability to quickly respond to the input with the gain change.  However, I have found some filter functions that spread the quick transient intermod further and become totally inaudible (this is a nonlinear filter function vs. time and helps quite a bit.)
Working in the domain of the linear (absolute value) detection instead of the square law works/sounds different from everything that I had worked with before.  By directly using the absolute value, the responsiveness of the filters is much quicker (with perhaps a bit more distortion -- the intermod that I spoke of above included), and very helpful for volume expanders.  For compressors, square law does work better -- even though the responsiveness is slower.
So, after very careful lining up of all of the attack times, utilization of the 'notch' trick, and also using the dynamic nonlinear filters (basically they create a symmetrical decay time based upon signal dwell -- giving fast response but don't propagate the glitches as badly into audible noise.)
I'll be release a very, very improved version of the 4 band psuedo DolbyA with 99% of the raspyness fixed.   Sometimes, a nice, soft timeconstant is destructive, and other approaches might need to be taken!!!

Results of the VERY improved code reside in the repository:

The ABBA voices don't intermodulate as nasty now.

I'll be posting the new pseudo-DolbyA within a day or so -- I have had to include some of my 'no-desire to release' technology to filter out the hash from the gain control -- it really does help!!!

John Dyson

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #57
New, vastly superior pseudo-DolbyA decoders now available (source, Linux and Windows).  On Linux, there is sse3 and avx optimized code (don't have a Pentium 4 version ready yet), and for Windows there is an Atom optimized version that should work on recent Atom and i3,i5,i7,i9 CPUS.   The Linux versions should work on recent CPUs -- for the Atom, one must use the SSE versions.

New improvements -- vastly superior level detector, much better than then two level detector used in the original DolbyA design, but rather very high resolution for the attack times.   This minimizes intermod on the attacks, and *might* perform better than even the original hardware.
The speed is much faster now -- I removed the attack/decay enhancer -- which is more useful for normal expanders, and not when emulating an existing design.

Source code now provides some hints as to some of the intermod reduction & dynamic attack/decay technology that I had created.
The source code for both Linux & Windows resides in   The only difference is the Makefile.
The binaries for Linux reside in da-linuxbins.txz, and the binaries for windows reside in   Note that the dlls
must be in the same directory or in the search path for the windows da-win.exe to work.
The usage instructions are identical to previous versions.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #58
Good news and bad news -- the good news first:  the previous version worked OK.   The bad news (or  good, however you see it): these new versions are better yet.   Here is what happened -- I was trying to speed up the operation by removing my transient enhancement code.   Actually, it isn't really 'enhancement', but appears to properly manipulate the attack/decay so that it better matches the old HW implementation.   Even though I continue to use the transient improvement on the 'restoration' software, I thought that the psuedo-Dolby would work okay with the simpler/faster code.  Again, it does work both ways -- but there are some end cases where certain kinds of distortions created by a DolbyA encoder aren't undone UNLESS using the more advanced transient 'enhancement' code.   If you listen to a DolbyA encoded version of Rainy Days & Sundays from the Carpenters (and there are more of them out there than you might think), you'll hear the removal of the crackles in Karen's voice when you use the better code (which I have reluctantly provided here.)  I made a few mods to speed up the transient recovery mechanism (it resides in recursexp.h), and it seems to do okay -- obliterating the crackles in Karen C's voice.   So, here are the versions that have more complete capability:

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #59
Here, I have produced two snippets...  The first is the obviously DolbyA encoded version of 30sec of Rainy Days & Mondays, the second is a fully decoded (with the da software also using the restoration processor...)   (please don't expect to get this sound with just the psuedo-DolbyA decoder -- but it does a critical and necessary part of the job.)   If you listen CAREFULLY (and it is difficult to distinguish), but the crackles that I have talked about are removed...   If you listen to the 'rainy' at near the end of the example (about 22 seconds in), the crackle is turned into a fairly correct crescendo.  (It is difficult to distinguish unless you set the level very carefully.)  Again, without this most recent update, I couldn't remove the crackle. 
damondays-noproc.mp3 is the raw DolbyA encoded version (and yes, I got it that way)
mondays-proc.mp3 is the cleaned-up version.

The frequency balance of the 'cleaned-up' version is quite different from the DolbyA encoded version -- note the HF enhancement in the DolbyA version.

Regarding the availability of DolbyA encoded material -- listen VERY carefully to what I have provided as a DolbyA example.  You'll notice a relatively high level of background hiss and a little bit of high frequency emphasis (similar to, but not the same as DolbyB.)  The big difference (in listening) between DolbyA and DolbyB is that DolbyA compresses a wider freq range.

Now -- another odd thing is that (in this example) I didn't use a normal mode for the psuedo-DolbyA, but I used a modified version (as had been suggested to me by people from history.)

Firstly, you CAN decode the music with the raw, straightforward pseudo-DolbyA decode, but there is something else that they have apparently done (after some reverse engineering by me) that they are doing both a L+R and M+S DolbyA compression on high frequencies.   Apparently, they are doing a standard L+R DolbyA encode with an extra M+S DolbyA for the top two bands only to enhance Karen C's voice.

So, to decode this song -- I am using a M+S, then L+R HF only decode (using DBAMODE=1 on the psuedo DolbyA decoder), then use the restoration processor that easily takes care of the two low bands of DolbyA decode plus some more expansion at HF.  (Even though the restoration processor has a VERY different architecture, the restoration processor does a reasonable approximation of DolbyA decoding with some major caveats -- but it is better to use a more correct DolbyA approximation when possible.)

Note that the restoration processor (unlike most expanders) DOES NOT SKEW THE FREQ RESP!!!   This is one reason why I don't call the restoration processor 'an expander.'  The processor itself is still a little in flux, but it is slowly, but surely getting cleaned up and producing better and better sound.   I plan to release it very soon, when it wont be an embarrassment and I wont be making mistaken releases like I just did with the pseudo-DolbyA.

Full song is at:

John Dyson

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #60
Quick note -- the files in the repository just happened to be the wrong versions...  They weren't too far off, but there was an error in the production of the mp3.   I UPLOADED NEW VERSIONS OF SOME DEMOs to the repository.   The versions that I uploaded to this site were correct and I believe accurately show the improvement. You might want to check again if you already listened.   Another case where the harshness is removed is in 'Reason to believe', where some of the voice crescendos(sp?) have no distortion and are reasonably reproduced.  Most versions that I have recently heard have a kind of ugly distortion of some kind (probably compressor or DolbyA encoder artifacts.)  The processors do get rid of most of that kind of stuff.

I am also JUST THIS MINUTE working on the restoration processor -- cleaning up some of the overdesign & hacks.   I think that I am going to separate the compressor and 'restoration' functionality -- and MIGHT even distribute the compressor sources first since that code is a little more mature.   I haven't written much about the compressor recently just because the expander/restoration part of the software is usually more useful to audiophile types.

When utilized in cases where a compressor is really needed -- it is generally not very obnoxious sounding -- but does sound like a compressor/limiter.  SOMETIMES I actually have to check to see if it is operational.  The compressor has various features like a totally automatic attack/release scheme, where the release is 100% mirrored from the dwell time of the signal.  Also, there are several forms of intermodulation reduction (some are partially redundant.)  I don't know if I am going to make the release time adjustable (it does a good job automatically determining the ideal release time), and the attack time is also carefully crafted to avoid intermod -- yet has apparently instantaneous attack.   I am just thinking that sometimes it is expected to be able to tune the release time.  Most often, if the release time is forced to be a certain value -- it won't sound as good as the compressor/limiter can do for itself.  Of course, the threshold, input gain, and compression ratios will be adjustable.   The eventual configuration will probably be stripped down to a 3band on input, an 8 band, a prelimiter (which is basically a moderate compression ratio single band), and then a limiter.   When used in combination (where the 8 band is optional), the results are pretty good -- and it is simple to use.  Each stage sounds good even at over 10-20dB of gain reduction -- but usually that kind of gain reduction is more trouble than helpful.

However, the highest priority is making the restoration processor code look better, and fix nits whenever I find them.  It is NOT complex code, but utilizes lots of vector operations (SIMD instructions), which often require a different kind of programming technique -- and sometimes the code is a bit odd looking.   Just trying to make it more presentable.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #61
Just for fun -- I produced two dynamic range compressed files derived from 'rainy days and mondays' and 'reason to believe' on the repo site.  These files are dynamic range compressed with a fairly close-to-release version of my compressor.   It is a true RMS/DB-linear compressor & limiter system.   It has a 3band, an 8 band, a prelimiter (which is just a very safe single band), and then a true limiter (plus there is a clipper, but very seldom -- if ever -- becomes invoked.)

The demos have had NO tuning, and the source material were the exact same processed (restoration processed) mp3 files on the site -- so they had quite a bit of dynamic range.   Also, these files resulted from several generations of mp3 conversion (I originally have pristine flac DolbyA files to start with, but would have been more work to regenerate a flac version to start the test.)

The compressor should sound pretty good to most people (as compressor/limiters go.)   This has my latest anti-intermod technology, and has fully dynamic decay time (with carefully time attack times for minimal intermod.)   Frankly, the attack time is not a typical RC generated timing, so it is hard to claim what the attack time is - but it could be described as anywhere from instantaneous to 2.5msec, with probably an average of 500usec to 1.5msec as the actual amount most of the time.  The total effective attack time (for the entire chain) IS instantaneous however.  The decay times use a combination of fixed decay, input dwell time mirrored decay time and dB linear decay (depending upon the phase of the decay slope.)   There is always a minimum decay which is directly related to the period of the audio waveform (actually ends up being an approximation to a multiple of the period.)

The audio detection is a combination of a RMS and linear calculation, where the detection itself is a dynamic attack as described above.  Because of the attack shape, the intermod transient is minimized (not totally zero, but as good as one can get with a sampled system.)  Most of the rest of the gross levels of intermod are removed by specialized filtering.

Much of my testing was done with a combination of music styles (and test signals) which excited some 'troublesome' results without careful design consideration.   This is one reason why I have used ABBA alot -- they have two female voices with a similar pitch (a little different, but similar), and their voices can easily produce some ugly stuff if processed with a nonlinear system (like a fast compressor or limiter.)   If the ABBA voices pass through fairly cleanly, then that is one of my test successes.  (note that it is easy to pass such material without noticeable distortion, but the solution is to use long attack/release times -- my processing is for all practical purposes instantaneous, so there is a lot of opportunity for intermodulation at high audio frequencies -- thereby POTENTIALLY producing nasty, ugly sound.  I chose NOT to use the easy way out.)   A good example of such intermod (a gross amount) is the beginning of the ABBA song SuperTrouper on most releases (precious few releases of that song have clean rendition of the beginning.)   My 'restoration processor' does produce near-perfect results on that material.  *Below I posted 5 second start of SuperTrouper to show the intermod (recent US CD release) and no-intermod (the restoration processor) versions.

So, this is just on a lark, also since I mentioned the compressor on a previous post.   The example files are now on the same repository as the 'restored' versions of 'Rainy days and mondays' and 'Reason to Believe'.   Note the word 'compressed' in the filenames for the re-compressed versions.  These compressed versions were NOT renormalized, so you can also see how well the limiter works -- the output of the prelimiter would hit the limiter a little bit once in a while, but the limiter is essentially noticeable distortion free at least at 6dB of gain reduction or beyond. (Controls are placed on the release times so that a deep release doesn't accelerate the decay.)

Site with the Carpenters examples mentioned here below again:

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #62
Made some improvements on restoration software.  My high frequency timeconstants were a little too short, and caused a fairly much harsher sound than what is really desirable.  The timeconstants cause a tradeoff between noise reduction, distortion and noise modulation.   I was biasing too much to minimize noise reduction/noise modulation, rather than minimizing HF distortion.   The restoration processor (and entire complex) already has so much NR that it really doesn't need as much as it is capable of -- so I pulled back the HF timeconstants (the detector release time constant used to be approx 2.5msec, and now is closer to 10msec in the linear domain.)   The actual release times can be significantly quicker due to the transient recovery processing, but the basic average release time for the detector is a very important number.   As a result, I re-generated the demos on the repository site and also re-compressed some examples (look at the filenames for the word compressed for additional compressed versions.)  Most of the compression examples would be fairly heavy record distribution or almost high-quality FM broadcast.  I also produced one compressed version which is closer to how I'd produce a disk/CD (RECORDING-compressed is in the name.)

Anyway -- the sound is quite a bit sweeter now.  This is one reason why I haven't released the restoration processor yet -- even though the architecture is correct, and most of the parameters are very close to correct -- there is still some tuning to be done.  The compressor/limiter just went through some vast simplifications & improvements and is coming much closer to releaseable.  There is one architectural matter on the compressor/limiter WRT the freq response balance on the multiband compressor.  I have achieved automatic freq response balance on the restoration processor, but still dont have perfection on the compressor/limiter yet.  Otherwise, the compressor/limiter is amazing from the standpoint of fitting the contour of the music without needing attack/release time tuning.  For the repo location again:
It truly is easier to use the repo now -- I made sure that the format of my mp3s match what they want, and you might be able to directly play the examples now!!!   My previous mp3s were too high quality and they didn't accept them fully, so I am using lame -preset extreme -V2, which gives generally just under 200k/sec (which they appear to accept better.)


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #63
Just uploaded a new version of the examples (again -- sorry.)  It has been very difficult to finally tune out the HF distortion while avoiding the modulation-type (or pumping like) noise.   I have extended the post-transient-improvement release times (for the 2.5k, 5k and 10k freq ranges) as far as I dare.   There is a user accessible tunable setting which can back off some of that extension, but as long as there is no noise pumping, then it is a good idea to keep the extension fully enabled.

Also, there is a rather 'different' or 'odd' attack/release dB domain scheme -- where the filter* actually does a 'leveling' rather than seeking a target.  So, the gain tends to stabilize if the signal level (at each band) is relatively stable.  So, if the signal has quick level changes, then those are followed at the peak available attack/release rates (rather they be in the dB or linear domains), but if the signal stabilizes, then the effective (most of the time) release time increases (or the rate in the dB/sec sense decreases.)  There is little need to play with the gain at full speed all of the time, and letting the gain settle (in a relative sense) is only a good thing.  Right now, this effect is not very strong, but I'll be working on something that might be more helpful -- if it is helpful at all.
    * I might use the term 'filter' in some cases where it might be confusing to an Electronics Engineer -- the 'filters' are a group of C++ classes which support various attack/release schemes.   These 'filters' are pretty much plug interchangeable -- for example, there is a dB rate filter for N freq bands, a dB rate filter for 1 freq band, a linear rate filter for N freq bands and a linear rate filter for 1 freq band.   Along with those relatively standard filters, there are dynamic attack/release filters which have numerous different kinds of characters.   All of these filters support directly or indirectly an adaptation to differing sample rates.  So, this allows the various audio processors to more easily adapt without doing sample rate conversions on the input and outputs.  This 'filter' scheme supports easy experimentation in a singular framework.

John Dyson

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #64
In my numerous (literally 100's) of tests of the pseudo-DolbyA expander, I have been getting a consistently odd result -- it seems like the 9k-20k output is over enhanced by a variable (changing gain-like) amount.  After reviewing some of my math that converts the feedback to a feedforward design -- specifically for the oddly configured high two bands in the raw DolbyA design -- it appears that my math has an error.   It isn't in the 3k-9k range, but the error appears to be in the 9k-20k range.  The reason for the error is my laziness of thinking, but the complexity in the design is related to the combination of the 3k-20k and 9k-20k measurements which then control the 3k-9k and 9k-20k ranges (note the overlap in the measurement, but the separation of the actual control.)  This wouldn't be so weird, except this is also a feedback gain control, but I had to unfold that feedback into feedforward.   After careful review (and after consistent irritation in my listening), it appears that I was indeed miscalculating the gain for the 9k-20k range.   I'll be producing a new version of the psuedo-DolbyA software  soon (I actually have it working now, but requires significant testing and rebuilding on Windows platform.)   It sounds prettier and has less of a harsh HF (9k-20k) sound.  The improvement seems to be better than a decrease in fixed gain, but rather the gain control sounds smoother.
I'll post a new version ASAP, but for those with source code and willing to rebuild:  at about line 2057 in audioproc.cpp, there are two lines that look something like this:
      VFP4 pgainl3 = (pgainl[3] + pgainl[2]) * VFP4(0.562);
      VFP4 pgainr3 = (pgainr[3] + pgainr[2]) * VFP4(0.562);
these need to be changed to:
         VFP4 pgainl3 = (pgainl[3] * pgainl[2] * VFP4(0.562) + pgainl[2]) * VFP4(0.562);
         VFP4 pgainr3 = (pgainr[3] * pgainr[2] * VFP4(0.562) + pgainr[2]) * VFP4(0.562);
I have not tested carefully enough yet, but I have a good 'feeling' about it.
Again, I'll post fixed versions ASAP.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #65
Good news (after mildly bad news.)  I have produced a corrected version of the psuedo-DolbyA.  The very high audio frequencies sound significantly clearer.  There was always a bit of a harshness that propagated through everything after using the 4 band pseudo-DolbyA decoder.  I was always attributing it to the primitive nature of the design, but other music that was correctly decoded with commercial equipment didn't have that 'harsh' sound -- so it must have been something wrong with my design.  I think that this code that I am posting here is an improvement, and didn't require a lot of complex changes -- but only correct a calculation error for the band 3 (9k-20k) range.  The HF(9k-20k) gain is different than my original code, and it will also change differently.   This difference in change seems to have linearized some of the apparent HF intermod or distortion that kept on frustrating/irritating me.

Along with the postings of the corrected programs (and sources), I have also re-uploaded some of the music examples on the repo sites.  The only examples changed so far are the Carpenters samples.   They really do sound better.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #66
I have a few quick suggestions on using the psuedo-DolbyA.  I have previously only given very rough guide, but here are some real world examples:

If you want to do decode a DolbyA recording (many which are compressed and have a slightly intense high end might be DolbyA encoded):
da-win --cmd="(l,-1.5,0.0)" <infile.wav >outfile.wav
the '-1.5' above is the DolbyA threshold.  I have found the useful range (revised from before) to be between -6.0 and +3.0, where typically it is between -1.5 and 3.0.  If the music seems like it has hard sounding HF, it often means that the threshold is set too low, so you might try increasing my 0.75 or 1.5dB.  If there is odd pumping, it is usually too low, so you also might try setting it higher.  If you feel like the music is too dull, then try decreasing the threshold.
If you have music that still sounds 'hard' sounding with too much HF, then it might have been compressed again with dolbyA or an old-style FET compressor to boost HF.  Here is a trick to get rid of that (after using the basic decoding operation above -- pass the results from above):
da-win --cmd="(l,-1.5,0.0)(m,-1.5,0.0)" <outfile.wav >outfile2.wav
This will add a layer of anti-esss sounding to the voices.  You might just need the 'l' command in the cmd switch, or you might need both the 'm' command and the 'l' command.  (l means L+R, m means M+S.)
In some cases, just the anti-ess can result in beneficial quality improvement, with the LF/MF compression will persist, but the HF will be brought better under control.
The da-win or da-avx programs can run in real time in a pipeline using sox or whatever if your computer is fast enough, and if you have a minimal number of subcommands for the expander to do.  (Each item in parens is a subcommand.)

If you want running information display as to levels (it can be interesting, and also help tune the threshold), use the --info=1005 switch.  Here is a complete decode command while using the --info=1005 switch -- this time with a zero threshold, and 1dB gain before decoding:

da-win --info=1005 --cmd="(l,0.0,1.0)" <infile.wav >outfile.wav


Here is the general scheme for the 'cmd' switch:
--cmd="(<subcmd0>, <threshold0>, <inputgain0>)(<subcmd1>,<threshold1>,<inputgain1)..."

where subcmdn is l, m, or b .   l means L+R expansion, m means M+S expansion, and b means in-between L+R&M+S.

thresholdn is the DolbyA threshold...  Without a tone (which the program cannot use anyway, because it isn't really DolbyA),
you have to guess about the threshold (usually between -6 and +3.0) units in dB.

inputgain is the gain before that stage.  Sometimes one might need to make-up lost gain before previous stage.  Usually, when done in cascade, there was no gain between DolbyA units, so you might as well try '0.0'.


The 'm' or M+S trick is something that appears to have been used from time to time in the old days to get better NR and/or to get special audio effects without appearing to be very compressed.
I found that the 'b' command also to be useful from time to time, however it is quite experimental (as is the entire program.)

Another example -- if you are using the sox program (and associated) on windows or on Linux/FreeBSD or whatever, then you can
do this if your CPU is fast enough:

sox infile.flac --type=wav - rate -v 96k | da-win --info=1005 --cmd="(l,1.0,0.0)" | play -q -

The above will take a flac music file, psuedo-DolbyA decode it, and play it on the standard output device.   The sample rate is forced to be 96k.  You can use 48k if your computer is slow, and you want to be able to try to run in realtime.  44.1k doesn't give much benefit over 48k, but might work (and the quality will probably suffer -- absolutely no testing at 44.1k other than quick verification.)

If the input data type is not supported by da-win (must be 16bit signed or 32bit float, 44.1k, 48k, 96k or 192k), then you can specify one of the supported formats to force the mode thus (btw, 96k or secondarily 48k are preferred by da-win or da-avx):

sox infile.mp3 --type=wav --encoding=floating-point --bits=32 - rate -v 96k | da-win --cmd="(l,1.0,0.0)" | play -

In the example above, the input file type is mp3, converted into a 32 bit floating point at 96k, the decode display is disabled (no --info switch), but the default display for the play command (subset of sox) is enabled by default.
So -- the above are some examples that might make the program easier to use.   I had waited to write this until I am satisfied as to the sound quality.   I am now working on finishing the cleanup for both the restoration processor and the 'super compressor/limiter.' :-).  hope to make those available within days/few weeks.

Please contact me if you have an application and wish to adapt/apply this program and want a little help.  I am more than willing to help if you want to use this program, and also willing to make reasonable changes/improvements if desired.

John Dyson

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #67
I came back to the 4 band (pseudo-DolbyA) expander, and learned another fact -- after 20+yrs working on compressors and more recently expanders.   I am offering this info to other developers (and of course, any  users) on this site.  Basically, another major improvement related to linear domain filtering of both the control signal and audio signal.   Please read further...

I kept on hearing a bit of graininess which could be partially mitigated by various tweaks like changing some of the attack/decay parameters and/or changing the delay between the AGC control signal and the audio signal.  In the last few wks, I have most of the settings set for ideal parameters, and been tweaking the high frequency (9-20k) channel, yet still getting some graininess.  The older (pre today) results sound better than the A/B comparisons that I happen to have (DolbyA encoded ABBA and various distributions), but still my intuition is telling me that something has been very wrong -- but not severe impact.   This matter has languished for a long time -- after continual improvements in adjustments -- it seems like I hit a 'graininess-minimum', but IMO, not good enough.

I had a BREAKTHROUGH again -- something that I had been thinking about off/on for years, but finally implemented it correctly:  The mixing (multiplication) of the audio and control signals produced aliasing components.  I already knew this -- but thought that I was doing almost as much as possible (adding some pre/post filtering, and controlling the attack/decay times), so the aliasing was the minimum those approaches could take -- but still some edginess was getting through.  *when multiplying two signals at say, 10kHz audio and 5-20kHz gain control and the multiplication is nonlinear (which it is), then you get all kinds of mixing products just like RF stuff -- 5kHz, 10kHz, 15kHz, 20kHz, 25kHz, 30kHz (just for the 5kHz control signal with 10kHz audio!!!)   This means that there will be ALIASING injected into the audio stream (because the sample rates cannot support all of those various by-products.)   Even filtering the aliasing after it has been generated is never 100% effective.

The mitigation (still not 100% fix) is to limit the spectrum of both the audio & control signals, but while doing the filtering -- make sure that the two signals stay synchronized.   Make sure that the spectrum of the ACTUAL control signal is limited appropriately -- must be done directly on the multiplication gain modification operation.

SO, the new version of all of my AGC/expander/pseudo-DolbyA will have a tight 20.5kHz linear phase LPF audio filter before doing the gain control, and also a 1kHz to 20kHz linear phase LPF filter for the control signal.  I have been doing various bits of testing, and I have centered on 1kHz or 3kHz LPF control signal filter for the psuedo DolbyA.   The sound is smoother, and there is much less of the low level 'grind' in some music.  Esp ABBA is much clearer (with the intense high female voices mixing together -- really tough in some cases.)  Some other examples on other kinds of music are also immediate improvements (again.)

There is significant improvement when using just a 20kHz LPF on the control signal, and I might used that on the limiter stages on the AGC/limiter processor for fast response  (and use the slower 1-3kHz filters on the AGC phases of it.)   For the expanders -- it is looking like 3kHz might be the best tradeoff -- where I hear only a very slight improvement when using the 1kHz over the 3kHz, but the attack times worry me for the 1kHz (300usec is okay in most cases -- even in some limiters, but 1msec is too long in some cases.)

I was pretty much amazed when doing a test with 20kHz LPF on the control signal -- and there was more improvement to be had with more of an LPF for the psuedo-DolbyA.   Frankly, might be able to get by with a higher frequency cutoff than 3kHz, but that is fast enough for the pseudo-Dolby (the fastest attack time needed is apprx 1.25msec, and 3kHz can provide 333usec.)

I'll be posting a new psuedo-DolbyA in a few days after much more testing.   There are numerous other little cleanups and improvements in the psuedo-DolbyA code also..   I have eliminated the need for using environment variables for specialty applications, for example.  I am also retrofitting every piece of gain control audio software that I am currently working on.

John Dyson

Truly beautiful/improved version of the psuedo-DolbyA 4 band expander.

Reply #68
Per my previous post, I have finally run about 75% of my tests with no unexpected problems -- so I am posting the results here.  Also, I am providing a pointer to some very-difficult-to-cleanly-process music, and I intend to add some more through tonight.  Currently, the demos are just ABBA -- but they are damned hard to process (two MF/HF voices that mix very evilly together -- very vulnerable to intermodulation in hardware or software.)   All of my previous versions of ALL of my expanders and compressors had no significant true bandwidth limit on the gain control, so the gain control would have both significant (excess/unneeded) HF components and also beyond-nyquist components.   The nonlinear multiplication between the gain (some in dB plus other nonlinear math) and the signal also causes some aliasing.   By tightly restricting both the audio signal bandwidth & the gain control signal -- AND CAREFULLY KEEPING TIME-SYNC & PHASE-SYNC BETWEEN THE TWO, the results are clearer and have significantly less harsh sound.  Currently, the control signal is limited to 3kHz BW, which is approx 0.3msec, while the fastest attack time needs to be approx 1msec, so the BW is sufficent and shouldn't significantly interfere with the dynamics.   Initial tests showed that even a gain control limit to 20kHz was an improvement, and I did try 1kHz with perhaps slightly smoother results.  However, with 1kHz, it will almost definitely interfere with the required attack time.   With the control limit set to 20kHz, there is excess control signal bandwidth with NO advantage, but the disadvantage is that there will be more aliasing (probably not a lot, but enough to probably notice from time to time.)
There is no real CPU difference between my 1kHz, 3kHz or 20kHz filter implementations, so the choice is purely based on sound quality.  Also, my filters are currently 'too long', but I used the length of 1544 because I have it in my library for quick access.  I know that I should calculate the filters on-the-fly, and will probably add that capability in the near future...  Currently my full expander/compressor project has approx 1000 filter files in it's library -- would be nice to just calculate.  I intend to decrease the filters to 256 or 384 taps, which will speed up the filter by over a factor of 4 (cache+raw CPU improvement.)

I am posting the latest (hopefully final for a while unless new bugs found) version of the expander (linux src/bin, win bin, win src), and a new 'readme' that might be helpful.   I am still too lazy to write a proper man page or manual at all.

The repository with vastly improved prettier sounding music is as follows:

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #70
I just re-processed (again) the ABBA examples as of the time of of this posting.  I was a little premature in talking about the results of the latest processing software -- but the current uploads are pretty good.   SOME of the ABBA example nditions are better than I have ever heard anywhere (e.g. UnderAttack & Dreamworld.)   I found most of the magic needed to get Dreamworld cleaned up (the version that I started with sounded like it had been run through DolbyA compression three times in sequence -- really, really bad.   These new results are pretty good in comparison.)  I didn't need to go to such extreme measures, however to clean-up Dreamworld as much as I had.    There are some other (non-ABBA) pretty good examples on the site also.  It is just that most other music isn't quite as challenging to process.
I have to be honest, however -- I made a few small improvements to the DolbyA which I doubt are very audible -- however, I did them as experiments.  If/when I make the next version of psuedo-DolbyA available, the modest improvements will be in the code.

Even though most of you'all are probably not ABBA lovers -- some of the short examples sound better than I have ever heard before.

Always -- be careful about dynamic range -- some of the songs start off very low volume and the hiss as a hint is minimized.

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #71
Have perhaps some useful information in using the psuedo-DolbyA 4 band expander.
Firstly -- it is incredibly likely that there is significant old material out there in distribution that is either DolbyA encoded or similarly compressed.
Secondly - I have some more evidence for that likely fact, and also an example script that I am using.

Demo on the repo site -- the Carpenters songs (currently the songs with names with two digits leading) have only been decoded with the psuedo-DolbyA (no games other than a little sweetening.)   Absolutely no use of the 'restoration processor' and its adaptive features -- purely the fixed time constants.  If these songs weren't compatibly encoded, the sound would be jumping around really bad -- but the results are fairly stable/normal sounding.  (The restoration processor can 'sort of' adapt to non-formally encoded material, but the pseudo-DolbyA decoder is very picky.)

I have an archive of a lot of Carpenters music that I purchased as an anthology many years ago.  I had always thought (similar to, but not as intense as my ABBA stuff) that there was 'something wrong' with the recordings.   They were too compressed and a little harsh on the high end (please note that I have no hearing above about 12.5kHz -- even though very solid to 12kHz, so what I might believe to be a 'little harsh' might be 'terribly harsh!!).

ONE NOTE:  I removed the 'sweetening' to make my demo more valid.

My initial results are showing that using the 4 band sidechain compatible expander (what I sometimes call the psuedo-DolbyA decoder) that practically all (I believe all) of the Carpenters music in the anthology are likely DolbyA encoded.  I needed to adjust the 'threshold' to -2.0 dB using the decoder command line to get proper results.  If the setting is off by more than about 0.5 dB, there are surging issues and frequency response anomalies that aren't terrible most of the time, but can work poorly on some kinds of music.
* I added a song that is not apparently compatibly encoded:  NotDolbyAEncoded.mp3 on the repo.  It might not be 3way encoded and just one way, but still you can hear some surging.

So -- my test is something very aggressive, and if the settings are wrong as I mentioned above, the sound is far from optimal.   I am running the 'decoder' through three phases (one might not be needed), where they are L+R, between L+R/M+S and M+S.   I think that only the two L+R, M+S are correct, but using all three does more NR and does sound good.

On some example songs, music like solo pianos are highly problematical, and if the settings are wrong, then it sounds bad.  Often, the recordings are hissy, and using the decoder helps alot.   The HF sounds a bit retarded until there is true HF in the material, and it appears very cleanly (especially now.)  I don't have a good piano example ready right now, but hope to have something ready soon (and will note it on the repo site.)

Here is the script that I am using.  Note that it is a BASH command line normally used on Linux/FreeBSD or whatever, but equivalent BASH command line can be used on Windows if you have cygwin64 or other packages installed.   Below is EXACTLY what I am using to do the decoding for the Carpenters songs on my repo (as I have noted above.)  For the Carpenters ONLY, I am ONLY using three passes of the 4 band expander (psuedo-DolbyA), the sox command and the lame mp3 encoder.  I did a little EQ sweetening -- but that is how the music is usually presented anyway on CD or LP.   The mp3 encoding isn't very good -- I looked at the spectrum, and not really happy with it.  The 24bit/96kHz flac that I am actually working with are a little better.

Here is the command line that does ONLY the raw decoding, without any EQ or mp3 encoding:

 dl=2.0 bash -c 'for i in /music/carp/*/*.flac; do echo DOLBY LEVEL: $dl; sox "${i}" --type=wav --encoding=floating-point --bits=32 - rate -v 96k | /usr/local/DolbyA/bin/da-avx --info=1005 --cmd="(l,$dl,0.0)" | /usr/local/DolbyA/bin/da-avx --info=1005 --cmd="(hb,$dl,0.0)" | /usr/local/DolbyA/bin/da-avx --info=1005 --cmd="(m,$dl,0.0)" | sox - --encoding=signed-integer --bits=24 --type=wav - gain -1.5 | flac --silent --force --best -e - --output-name="carpd/$(basename "${i}" .flac).flac"; done'

I have also included an attachment that is essentially the same, but is in shell script form (haven't tested that yet -- I have been using the actual command above!!!  The attachment is easier to read than the straight command line above!!!

John Dyson


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #72
More info -- I just ran some Carpenters tests again.  (results on the repo site, filenames end in "-2" for two passes.)   Instead of using three passes of pseudo-DolbyA, I more realistically backed off to two passes (instead of L+R, 1/2(L+R,M+S), M+S -- I am using just the more conventional L+R, M+S).
The sound is more alive when using just two passes (simply remove the 'hb' expansion da-avx command from the script), and set the threshold to approx -3.25dB instead of +2.0dB.
So, you'll notice that the examples with '-2' might sound a little better.   It is probably more realistic to just use LR & MS simply because of the slightly more detailed set-up required to do the 1/2 way matrix in real hardware.   This is probably what they used to actually encode the music (that is, the two passes instead of three.)   The 'three pass' might be best used to decode already encoded material when there is a lot of hiss that needs to be removed, but the 'two pass' does seem more alive.  It might be that the earlier material WAS encoded with the full three passes out of desperation to get rid of tape hiss -- because I did find that the later Carpenters' stuff (late 1970's and early 1980's) did not decode well with the full three passes either.

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #73
Sorry to keep pestering -- but info is coming fast and furious:

firstly, it appears that the ideal processing (after many experiments) is 'l+r, HF(l+r), HF(m+s)', or using the 'cmd' args, it would be: l,hl,hm.  The each of the three runs of da-avx (or da-win) would have --cmd args like:

Also, after doing the runs of the various songs with the above cmd args (instead of the ones specified above or in the shell script that I had appended above), I added another processing phase -- a linear one, absolutely no use of the 'restoration processor', but reordered some of the time after looking at some information about the self-correlation (that is patterns) in the music.
So, if you listen to the files with the -exp in the filename and compare with the others, it might be interesting.   Also, I am removing the files with -2 in the filenames, and when the files with -3 appear, those will be the music with the above commands (that is, again, full band l+r, HF l+r and HF m+s.)  The effects of the DolbyA compression just with the HF is to slightly sharpen the HF of the music.

Probably about 10 minutes after this post, the new examples with the -3 in the filenames will appear.  The -exp versions are already there.   I am also considering removal of those processed with the first version -- the new ones are vastly improved and sound much more musical.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #74
New version of the decoder.  Additional filtering on the 80Hz and 80-3k range.  I realized that I can filter the two low bands much more to remove some more intermod products (and mitigate even more aliasing) without impacting the attack/decay times.  Demos much more complete on the repo.   -exp files have a new experimental time domain decode.  All examples except for the Sergio Mendes examples just use the 4 band (pseudo-DolbyA) decoder plus some SOX tricks...   I added the restoration processor to Sergio Medes to resurrect some of the percussion.   I am basically demoing the simple 4 band decoder to show how much a relatively simple piece of software can undo a lot of encoding.   I do have one example on the repo which was obviously not DolbyA encoded in the same way as the others -- you can hear the surging that decoding an unencoded recording can create.   (Actually, the 'non-encoded' copy was 'less' encoded than the others...)  If you decode without 'surging', 'loss of HF components' or other artifacts -- it is quite possible that you are decoding a recording which is either DolbyA or side-chain compressed.   Undoing side-chain compression is just as helpful as decoding old DolbyA recordings.  (A full expander instead of a side-chain expander can easily do TOO MUCH expansion.   The 4 band expander is designed to undo sidechain compressed material.)

BTW, I fixed the default Makefile on windows so that if you have cygwin64 installed, the program should be easier to build.  (Before, the correct Makefile was an obscure file in the source directory -- sorry about that previous mistake.)

SimplePortal 1.0.0 RC1 © 2008-2020