Skip to main content
Topic: Have a working 'expander' based on DolbyA (not same design) -- works well. (Read 11265 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #100
Your terminology is slightly confusing.  It's not surprising for a studio tape since 1965 have been Dolby A encoded; it is surprising for it not to have been decoded for commercial release .  So your concern is really 'Dolby A encoded stuff released without decoding'.

AFAIK, Dolby A decoding was not available on any consumer playback gear (e.g reel to reel decks).  So no Dolby A 2TRK tapes would have been released commercially, unless it was by mistake.  Yet you say you have  "a 2TRK copy, unprocessed and is a DolbyA master of ABBA Gold and MORE ABBA Gold."    What gives you absolute certainty that these tapes are what you say? 

Given all the adjustments to EQ and dynamic range available to CD mastering engineers, I just don't see how you can be certain that commercial releases are suffering from Dolby A nondecoding, versus normal mastering choices.   Tipped up treble is hardly rare in CD masterings, and neither is compression, and both are easily achieved without Dolby NR playing a role.

Yet you go on to claim:
Believe it or not, often the better quality disks tend to leave the DolbyA undecoded -- it is the horridly misprocessed material that isn't usefully DolbyA encoded.

I don't see how you can *know* this.

However, you also say you have 'PROVABLY' Dolby A encoded material . If that is so, then you can do the comparison I mentioned previously  (decode with a Dolby A hardware decoder vs your software).

But there's also the whole issue of the Dolby calibration tone, which would only be printed to the master tapes;  without that, setting levels for decoded playback, is guesswork....

As I wrote above -- we have done careful comparisons between a real DolbyA unit (cat 22 based/360-361), and my decoder.  My decoder works generally better than the original DolbyA unit and also the other SW decoder (I am not competing with other -- dont need to do so.)  The biggest difference betwen my decoder and a real DolbyA is a smoother/cleaner MF/HF, but close to the same response.  The 'other' SW decoder doesn't sound like a DolbyA AT ALL, but is better than nothing.*  The gain curves between a real DolbyA and my SW decoder are almost IDENTICAL (within about 0.25dB most of the time, and 0.5dB error once in a while.)  The curve shapes are indeed the same (it took high order polynomials to get it that accurate.)  ALSO, we didn't just compare against one unit, but multiple units -- and the person doing the comparison uses DolbyA and DolbySR almost daily.  In fact, we are probably going to start on an SR project, even though less SR material appears to be leaking out.

  *In my work, I found that emulating the filters on the cat22 was futile, and probably what the 'other' SW decoder did -- I designed the filters based on another concept (keeping it secret), and it works wonderfully.  The cat22 filters (or an adhoc filter bank) tend to sound like the other SW decoder and not a real DolbyA.

The lack of tone is a definite issue, but I have found that approx 3/4 or more of the material has a required DolbyA threshold within 1 or 2 dB.  It is easy to seek out the best level (usually takes me 2-3 tries to get within 0.25dB -- which is generally close enough.)  Also, if there is a tone, it is just an offset from a displayed level to calculate the necessary threshold (problem solved.)
Most of the online/CD material seems to require a threshold (defined by my software) of -14dB +- 1.5dB or so.  There are a few out-liers which require very different thresholds.  One recording that I have confounds me by requiring a -7.5dB threshold, but almost all others are in the -14dB range.  Usually, within the same album, the thresholds are the same.

Almost always, 0.25dB error is close enough, and most material works well with 0.50dB error.  Music that seems to require very close setting would be things that sound like the master copy of UnderAttack from ABBA.  Also, Band on the Run is also a little bit picky in the pauses at the beginning of the song.   Gotta get those tricky time constants correct (and a simple attack/release time constant WILL NOT WORK CORRECTLY!!!)

Of course, for professional purposes, you really do want to get within 0.25dB, but for causal listening practically all of the advantage can be had with an error of 0.5dB.

The way that I have found to use the decoder is to grab my on-computer archives and decode them (producing CD or memory stick for later use).  In some cases, it is very desirable to run some finalizing on the material (after decoding, ABBA needs a little compression/limiting to sound correct.)  However, my DolbyA copies of ABBA do sound NICE when properly decoded.  In the specific case of my ABBA recordings, almost all of them are not usefully DolbyA encoded, but I do have one which was obviously not finalized -- and fortunately (for me) left the DolbyA on as the normal dynamic range control. 

As I have claimed -- just because I have a certain tool, it doesn't mean that I believe that the tool is useful for everything.  It is just that it is useful (whichever DolbyA decoder you use -- a real DolbyA or my SW) in more cases than most people might realize!!!

Got rid of a lot of 'earwax' from old DolbyA recordings.

Reply #101
As you probably know by now -- DolbyA recordings without decoding tend to be a little on he harsh side, and also have more old tape hiss than really should be presented to the listener.
Proper decoding can get rid of the harshness and definitely remove a fair amount of tape hiss (mostly forgotten about nowadays).  However, the DolbyA encoding process has a mechanism to keep the compressor from excessive excursions and also an intrinsic delay which on average tends to remove some of the natural brightness in the music.

So, without decoding, we have a somewhat clipped high frequency range -- boosted between 6-15dB, with lots of hiss.  (The boost is less at high signal levels, however.)  This does make the sound harsh.

I have improved the decoder further -- still in the testing stage, but initial results are superb -- the 'clipping' is still there because of the encoding, but the natural analog delay is now mildly compensated for, along with some mitgation of the more nasty effects of the clipping.  Cannot replace the removed information because of the 1msec clipping, but still there are cascade effects that are now gone (a true DolbyA decoder 'fixes' the problem by further damage to the signal and even more delay to hide the clipping.)  Some things in the DolbyA design in the encode/decode process naturally compensate for each other, but other things just get worse for each pass through an encoder/decoder.

Thanks to the digital DSP gods, if the signal defects are known about, they can more easily be compensated for.   The compensation in a real DolbyA might have required some kind of delay mechanism that doesn't exist in the design.  Back when DolbyA was designed, a loss of transients and/or some nonlinearity in the top octaves of audio wasn't such a big deal.  Nowadays, loss of ANYTHING audible is a big deal.

One thing that the listening community needs to be educated about -- a lot of the stealth DolbyA encoded listening material out there has TOO MUCH HF intensity and is actually distorted to some degree.  We (the listening public) have become used to a compressed/distorted high end on digital materials (esp on the old stuff.)

The DolbyA decoder both removes the excess intensity (while maintaining whatever true HF there is left in the recording), and now even helps to compensate for some of the encoding short cuts.  The decoder DOES NOT replace the missing transients, and unlike my 'restoration processor' or 'uncompressor' as I call it today, doesn't try to recover some of the transients.  Basically, this new version of the decoder uncovers much of the transient that is left in the recording without faking anything.

The decoder is most likely a better decoding device than has ever been publically developed, and is able to recover more of the music HONESTLY than ever before.   The result is a pure rendition of the before-encoding.

Historically, the phase of encoding that I am decoding is the tape that goes to the last phase mastering or to the pressing facility.  They, of course, tried to protect the material as long as it could be protect with NR systems like DolbyA.  In a way, it is very fortunate to the enlightened that there is now access to a really good digital DolbyA decoder.

Newest/latest greatest decoder (DolbyA compatible).

Reply #102
Here is a copy of the decoder and a nice description of what it can do, various music that it works on, and how to use it.  To install, just put the binaries and DLLs into the current directory and use it!!!  Works on recent Intel boxes.

It is lightyears beyond anything else that I have done, and works better than any decoder out there.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #103
Found one minor tuning problem (only appears in very specific cases) and also improved the anti intermod after the processing, the intermodulation products of each bad are better filtered -- if you use --hpf, it filters the intermod even better.  The only real cases where the intermod at HF is really irritating is on certain materials with continuous natural HF material like ABBA.  The results are a little smoother yet now... (The results were already pretty good.)

The tuning problem manifested most (out of all of my recordings) on "Band on the Run', where he stuttered timing at the beginning made an error in the attack characteristics more obvious.  The actual error wasn't in the basic attack code, but rather the intermodulation mitigation. This error has been fixed (at the expense of a slightly greater amount of intermod -- probably NEVER noticeable.)  If you want to disable most of the intermodulation mitgiation, you can use the --raw=257 switch.  That disables both the early and late intermodulation mitigation.  Most of the difference can be heard when there are multiple continuous voices or tones, and there'll be a sharper/rougher edge when the mitigation is disabled.  There are other negative effects of turning off the mitigation, and the goal is that there never be a real problem when it is (by default) enabled.

Sorry for this quick follow-up release, but i was running one of my critical review passes, and found the potentials for improvement. Everything else is the same.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #104
Just a quick post -- have a minor bug on the highest freq channel.  If it is a little hard, that is because I made a mistake in moving some code.  I am doing more tests, and intend a new version within a day or so.  I found another method for intermodulation mitigation -- running final tests, and will include the 9-20k fix also.  You shouldn't notice a problem very often -- when I saw the code (I couldn't hear the difference), I was quite irritated with myself.  I did such a good job of figuring out how to get 9-20k working perfectly, and then I made a mistake in the final calculation!!!  (IT has been correct in the past -- I had an #ifdef that I didn't move a line of code around.)

VASTLY superior decoder (decodes DolbyA) -- much intermodulation is now removed.

Reply #105
This decoder is not as good as possible, but probably better than normally available.  Beautiful results.
Previous version got really good feedback -- people starting to notice that the DolbyA encoding is real!!!

Filename containing the new -- really good -- decoder:
Short manual:  DecoderA.pdf


For convenience -- decoder posted here...  An example of a VERY DIFFICULT portion of a song -- lots of HF intensity, but intermod really suppressed!!!  (Not everyone likes ABBA -- but listen to the voices that are still seperate -- most copies of ABBA get smushed together.)  There are probably better examples -- but ABBA is about the most difficult to reproduce from my collection.


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #106
Found another step of intermod mitigation that was easy to apply.  Also, the previous 02may release was a little aggressive in removing excess LF when using --hpf. (excess LF causes some troubles in some cases.)

Also -- USAGE NOTE -- intermod can be further suppressed by using the --hpf switch.  This FURTHER helps to remove the troublesome out-of-band lower freq intermod in each of the 4 bands.  Previously, the --hpf switch was way too aggressive, but the new modification above plus the slightly modified --hpf switch makes another quantum jump (in the sense of BIG quantum, not SMALL quanta :-)).  The demo repository reflects the improved LF performance & behavior.  There are already some very non-intuitive approaches for improved quality.  Unfortunately, the next performance enhancement will most likely need to be optional, because it will require approx 2-3X more CPU.  The current version is approaching a quality vs. CPU perf limit with the current architecture -- I have some performance ideas, but I don't think that the cost/benefit (cost of my dev time) would be worth it.

Sorry for this continual iteration, but as the decoder is used more, there will be more improvements.  I suspect that without the next major (2+wk ahead) improvement that this should be stabilizing.  (I do another major step planned -- upcoming version should blow away ANY dynamics processing experts WRT quality -- -this should already be pretty good in the wider scheme of things.)

The updated version also resides on the decoder code repository.

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #107
Quick update -- decoder working great, but I have a usage suggestion:  alot of material has excess errant material in the upper frequency range.  I suspect that it results from previous encode/decode cycles and/or perhaps additional intermod being produced.  I have found that the intermod cascades very rapidly.  So, a minor suggestion -- if you are using SOX, apply a 'sinc -t5k -20k' to the output of SOX into the decoder/expander.  I am going to look into further shaping the attack/decay curves which might help, but it really isn't a bug -- if the careful attack/decay shaping hadnt been happening, even the sinc command above wouldn't help.

Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #108
More interesting news -- I have been thinking that there was too much intermodulation between the gain control and the signal.  It was 'GOOD', and in fact 'GREAT' considering that it is on a computer using DSP code instead of analog.  But -- I had a minor EUREKA -- I redesigned the attack/decay code (significantly) to better stitch together the two attack curves.  On a real DolbyA -- there are kind of two attack curves, but they are just basically one or another (this is the 'softening' or whatever he discused in literature.)  I reworked again -- already having two very smooth curves.  However, I decided to stitch those curves together so that there is NO discontinuity.  The difference in sound is significant (in fact, I no longer need to do the fancy approach that I expected to do in about 2wks!!!)  I am in the midst of very aggressive testing.

All I can say is that the excess sizzle is 99% gone now!!!   The suggestion about the sinc filter in the previous message is absolutely no longer needed on the next release.
The good news for me is that I don't have to do a big, long design of a complex filter for the 80-3000Hz band -- I was going to have to split it in two, but no longer needed.


Could not hold this back -- intermod really improved farther.

Reply #109
Up until now, I had been filtering out the out-of-band intermod products, but not trying to get rid of it at the source.  (I am speaking of the excess intermod, not the theoretically necessary intermod.)
I had thought that I had made the detection code as good as it could get, but I did another review.  It was indeed VERY GOOD -- pretty darned near perfect, but I noticed some kinks in the detection characteristic.  Ahhh.... NOWAY that would make a difference, right?   Well -- I stitched together the curves as well as I could do in a few minutes  -- no heroics, but just to make sure that the attack was smooth at the junction between the fast attack and the smooth gain control (DolbyA has two curves -- sort of) on each channel.
I joined the curves together perfectly, but also biased the joining towards the slower attack.  If you do the math, I think that one would find that the time domain joining and the level intersect when doing the bias between the speeds in the way that I did.  I did some listening, and it was close to perfect -- but then I did some more adjustments (basically compensating some other things for what I had done to the actual attack code), and magically 99% of the edginess is gone.  I ACTUALLY THOUGHT THAT THE EDGINESS WAS IN THE RECORDINGS!!!   But the sound is much smoother now, but everything is there.  Since I moved some of the timing around, I also had to better synchronize the audio with the control signal -- minor difference -- about 200usec movement in the audio signal.  That minor difference makes a big difference on how clear the transients and 'tinkle' things are.

I would NOT have done this release without it being a major improvement.  I was actually finished posting anything new for two weeks until now.  This is too good to wait.

Nothing has changed, except continue to use --hpf if you want to get the absolute minimum intermod -- some of that intermod results from the theoretical necessities of gain control, but doesn't really hurt the gain control that much when you remove them.  So -- it is good to use --hpf if you don't need 16Hz audio :-).


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #110
Been doing some tests on the decoder (very detailed), and made a few mods -- don't worry -- not worth an update yet.  However, there is more quality to be mined believe it or not.  I was SOOO amazed about the improvement based upon slightly changing the attack structure (decay has to stay the same.)  I am in the midst of writing some information on the DSPrelated forum, and hope to write the next installment soon (days.)  I am covering all of the various aspects of audio gain control and the things like different frequency bands, I'll be talking about attack/decay, intermod, dB or linear attack/decay.  Point being -- I have never been so suprised with this attack structure improvement.  It must be that so many of the details are 'perfect' that this small improvement is making a big difference.   I have a very fancy general purpose expander and GP compressor project also.  I think that I have new information to apply to the expander!!!
There are things that I am learning that are JUST NOT common knowledge -- all too often a compressor is designed with a very slow attack (slower than the audio frequency cycle time), and that limits flexibility.  I think that once I get it all together, I might be able to create a short paper on doing VERY good quality audio gain control.
This decoder (especially the current experimental version that I am working on) is doing some amazing things.  The experimental version has been more carefully tuned to fit the needed decoding characteristics -- and it is AMAZING how tricky it has been to unfold the 'inverse compressor' design concept of DolbyA into a feedforward and still match the attack/decay.  The attack is crazy-tricky because in the feedback structure, it has the initial 'surge' or 'boost' that the feedforward has to emulate.  Tricky stuff for sure!!!


New version again -- less aliasing yet, and also found more aliasing!!

Reply #111
I have been focusing ONLY on distortions of any kind in the last several days.  The general attack/decay matches every recording that I have in my position (that is DolbyA recording.)  Also, theoretically I can derive all of the general attack/decay parameters proving the reasons for each one.  The attack/decay is 'perfected' -- not playing with that at all.   Trying to get that 'sweet' sound.

More good news -- firstly, I found a minor bug that the postfilter wasn't being invoked sometimes unless one specifies that it isn't disabled -- stupid, huh?  Also, there has been an ongoing calculation bug for the 9k-20k range where it was boosted by a slight amount.  You'll find that 'Sess' sound is more pure now.  Also, aliasing is boosted less.  If you want that slight 9k-20k boost back, then just use the --raw=4096 switch, but I don't see any reasons for using it.  (I am keeping it only for regression testing reasons.)

I changed the attack filters (and the post filter) to produce fewer harmonics also.  The small nonlinear postfilter has been very slightly retuned to consider the non-ideal nature of low pass filters when the rolloff is close to the nyquist frequency.  This has an additional positive effect ofreducing the amount of useless HF (produced because of nonlinearity in the necessary processing) in the control signal which only produces aliasing.

Have gotten feedback and keep on telling me if you hear something odd!!!

The short document has NOT changed, so I am just posting the decoder zip file.


Oh my, got a complaint about decode quality on another forum

Reply #112
On another forum -- someone talked about the problem with decoding 'Band on the Run', but I knew about it -- blew it off.  I felt like that odd sound was the way that I remembered it when it came out (it had a bit of a duck at the beginning -- but it was 40yrs ago.)  Anyway -- I first blew-off the comments and simply 'fixed' it by disabling the MF decoding -- which is a feature that is available for when in the olden days that the MF compressor was disabled.  It was either due to failure or a quality issue -- but no matter what, the decoder has that feature.

However, back to the problem with the example...  I finally felt like I could no longer blow-off the problem.  I did a lot of thinking about the attack/decay characteristics and finally realized that I needed to compensate for the feedback-to-feedforward unfolding for the decay as well as for the attack.  I analyzed the situation, and found a reasonable solution (of course different than the attack solution), and definitely found out that there can be a decoding problem unless the compensation is done.

Anyway -- I am testing the decay code fix.  The good news is that the decay code is slower and doesn't quite have the intermod issues that the attack code does.  So -- it is NOT a big problem to fix.  However, the testing is not quick.

There will be a new version in a few days with this correction.  Several files which I though were NOT DolbyA encoded now are decodable.  So that excuse 'not DolbyA' is less likely correct now..

I am a little embarassed, but happy that I understand this better.  This DolbyA decoding thing is NOT trivial.


decoder: Bugfix for failed handling of uncommon dynamics

Reply #113
This bugfix is important even though it might only help a few songs that it previous worked with.  The good news is that it works on more examples -- the code was missing an important part of the math as explained below.

The bugfix only helps on very few songs that it could previous decode at all, but when it helps -- it REALLY helps.  The fix implements handling of better feedback to feedforward conversion.   I did an EXCELLENT job on handling the attack times -- I was focused because proper handling of such makes the sound purer.  The thing that really bothers me about a lot of the recordings nowadays is the terribly harsh sound, and I was focused on that matter.  I ignored the handling of the decay time, because I didn't think that it needed correction.  When I revisited the matter with seriousness -- I realized that I sometimes suck!!! :-).

This required a lot of testing as you can imagine, because changing such a basic behavior can totally break things, or break them in difficult-to-find ways.

The REALLY good news is that several of the recordings that I claimed might have been recorded by a broken or disabled DolbyA can now be decoded by this corrected version!!!

Finding bugs like this after such assertive statements by me has been humbling.  However, the good news is that the new version is available.  Also,  I have some examples from the group 'Bread' that clearly show how much of an EXTREME improvment can be had.
On the repository below -- listen to the two songs:

BabyImAWantYou and MakeItWithYou.

The examples are BEAUTIFUL examples of traditional recording techniques and REALLY REALLY show off the decoder!!!  The before and after are like night and day!!!


Re: Have a working 'expander' based on DolbyA (not same design) -- works well.

Reply #114
Minor update: possible fix/improvement coming.  Ran into some troubles with low level transients (that is stuff at lest than -6dB gain.)  FIgured out that I was mistakenly basing the long-term, feedback-to-feedforward calculations on the raw signal level and not the input-filtered version of the signal level.  It is a simple matter of changing a variable name as input to the calculation.  I looked at this all night, and realized the screwup.  The version that I am testing is what I had meant to do -- and this was in the section of code that was added to properly support the decay calculation.  I plan to make this available in a day or so -- running tests again.  It is becoming very boring to listen to music!!!

The basic decay constant is the same as it always was -- approx 30ms for HF, and 60ms for LF/MF.  However, consideration needed to be given for previous history because of the feedback configuration.  That support for previous history is what I added several days ago.   When I did the history support code, I used the raw/unfiltered (well, 1 or 2msec filtered) signal level and not the dynamic input anti-intermod filter results.

The primary result of this fix is that low level material (when running less than about -6dB of gain) will have a more correct decay time.  Before the fix, the decay time would have been a little too fast and cause some gating or L/R image problems at low levels.  The version in testing seems to mitigate the problem and results from a minor code correction and not a major redo.

Giving you the idea of how I made this mistake -- the delay calculation improvement that handles the FB->FF conversion required several hundred lines of code and is stuff that needed to be written FULLY from scratch (no textbooks, appnotes or research papers to guide me.)  So, even though I often write over 600lines of new, practically bug-free code per day  - that kind of programming rate usually only happens when doing something that had previously been well researched.

This current decoder programming IS the research!!!   :D .  I have NO idea as to the way to document the design, but planning to continue my recent DSPrelated posting to talk about compressors/expanders and how to make them sound good.  Might be able to talk a liltle bit about this decoder vs. a normal expander design.

I apologize for the error!!!


Promised decoder update

Reply #115
This one, if it works for you finally has gotten some serious tuning.  The decay code has now been corrected, but you might find that the threshold MIGHT have to be slightly increased, say from -15.75 to -15.50 or -15.65.    The biggest difference isn't really the corrected decay (better handling the FB->FF conversion), but also the intermod tuning is now really good.

The intermod calculations are not trivial, and there is a bit of tuning involved.  For example, I found that DolbyA sources tend to have a lot of 9kHz distortion built-in.  At the normal and higher settings, that distortion is carefully removed.  So, the decoder not only mitigates its own intermod creation, but helps with the encoder problems also.

There is a new switch, as documented on the DecoderA.pdf file.  It is --ai=<mode>, where <mode> can be none,off, min, med, high, max.

The default setting is --ai=med, which gives a good balance of negative and positive effects.  If you find some problems with the --ai=med setting, then I suggest -ai=high if you want to remove all but the worst intermod -- it might seem a little less crisp, but that crispness is 90% intermod.  --ai=off would be a setting that one might use if the crispness/fake HF might be beneficial.  I don't know a real reason for using --ai=none unless testing.

I have been getting hung-up listening to the music (at the --ai=high setting, which  I prefer) -- I am hearing things that I never heard before (like details in the ABBA chorus.)  I set --ai=med because I try to be conservative -- might end up setting 'high' as the default in the future.

Have fun!!!

Just posted decoder has a bug -- don't try to use it -- fix soon

Reply #116
Please don't waste your time on the recent version.  New one coming soon.  Bad HF freq resp problem, and my hearing has troubles sometimes.

Fix coming tomorrow.

Reissue of new version -- plus apologetic technical explanation

Reply #117
Firstly -- I apologize for wasting bandwidth and anyones time for my mistake.

okay -- here is the version with the HF restored.. Here is the technical explanation of the bug, and why it happened...  First, my hearing sometimes winks out on HF -- and I didn' tknow that the problem was happening.   The technical reason is as follows:  I had been using FIR filters that were like 1800Hz to 15000Hz bandpass, but the length was 511 at 96kHz.  I needed that 15kHz to really be 15kHz with no skirt, however, when also making a bandpass with the 1800Hz low end, it causes the skirts to widen.  The filter ended up encroaching on the HF region to below 9kHz, and any loss in that range is bad ( causing a characteristic limited HF sound.)  I didn't notice the problem till too late (hearing came back), and I panicked after distributing the broken version.

The correction was to seperate the bandpass into two separate filters -- I used a series of simple recursive high pass filters for the 1800Hz, even though the skirts are loose when doing simple filters like that.  For the HF side, I simply changed the BPF into a LF, and that tightened up the skirt at 15000 and restored the HF for the 3k-9kHz range.  (When talking about 3k-9kHz, the skirt on that filter is very wide, and so the passband for that filter extends well into the 9k-20kHz range.)  The way that the DolbyA compatible filters work are actually counter-intuitive -- where if the 80-3k band is active, it actually opens up the band well above 7kHz, and partially overwhelms the 3k-9kHz and even the 9kHz-20kHz range.  The VERY IMPORTANT band pass filter used to remove intermod on the 3kHz - 9kHz (and also the 80 - 3kHz filter also had troubles because of some special stuff being filtered there also) was badly messing up the HF most of the time.  (Sometimes the gain control would open up the HF band -- but most of the time the BW was limited.)

As i tried to describe above, the filters were re-architected -- with a compromise solution so as to avoid using LOTS more cpu to implement the LF range filters with an FIR filter.  I really need to use some kind of multirate scheme to decrease CPU usage, but frankly I worry about possible quality impact -- so for right now I am using simple textbook filter design.

The sound of THIS result is much more transparent, and even the --ai=max mode sounds fairly transparent most of the time.  I still recommend using --ai=med for pro applications, and --ai=high or --ai=max for listening (or producing tapes for listening.)  All modes using this verison of the code work MUCH better than the previous borked release from a few hours ago.

Sorry again -- everything that I am doing is IMMEDIATE and I am not holding anything back.


Last minute updated release. Wanna give you the best possible.

Reply #118
Update complete.  I keep on finding things that can/need to be improved.  This is going to have to be the best so far!!!

Each time, it keeps getting better and better.  Sometimes I make mistakes when making bigger changes -- this should be pretty good.


Bad day yesterday -- today, finally the results that I wanted

Reply #119
Have some patience.  I have been having troubles with one of my libraries, my hearing (fixed) and trying to get this thing sounding correctly.  I have also JUST NOW corrected my filter generating library -- amazed that it was working as well as it has been.  I caught one bug by running  a spectral analysis, and noticed a rolloff before I had expected.  Been going nuts trying to find the problem and then found that my FIR generation library (that I wrote) was borked (by me.)

This FINALLY sounds correct -- the way that I wanted.  I went for almost a full day breaking things yesterday, and I didn't realize it.  I go through day where my ears are messed up and finally they do clear up.  I woke up to a nightmare and been trying to fix problems.

ALL of my work is mathematically and engineering based, but the work has to be tested by me.  Unfornately, I ended up with a cascade of errors.  I found the culprit, and am even posting the source code for the corrected routines (that calculate the FIR filters.)

This version has been tested by GOOD ears (more than just me), and also I have some technology examples.  It is difficult to hear the difference depending on the material (some material is very obvious.)  Here are some instructions (only on this version and on) that you can disable all quality improvements and just depend on the carefully designed detector (more than just the typical RC time constant thing), but I didn't bother implementing the typical broken detector design -- so you cannot hear the worst possible from this software.

1)  The default settings are the best now.  No complicated options -- the default settings sound just as good as the suboptimal/unprocessed settings (just a little different.)
2)  To disable everything special, use the following:  "--ai=none --raw=8192"  (the sound will be crunchier on difficult material -- that is mostly the intermod.)

I PROMISE this is at least a pretty good version.  I was upset to see 14 downloads of the previous version.  I pray that people haven't wasted too much time.

I am back functioning very well now :-).  So is the software.  The culprit (but fixed) FIR filter coefficient generation library is also posted -- I didn't comment the code very well -- but to DSP people it is fairly obvious what it does.  To non DSP people - -the filiters require a long list of numbers that do things like low pass, high pass or other things like that.  These numbers are used by the general purpose filter code (kind of a filter program.)  This file 'makefir.h' generates those numbers based upon what kind of filter, how many coefficients and what kind of 'window' you want.  This primitive, but works for some cases of  what I need to write the code.

If you look at the simple DecoderA.pdf file, you might notice that I removed some of the fancy --ai options -- not needed anymore, because it always sounds good now.



Not replacing broken code -- more sound-sound improvements.

Reply #120
Finally -- more good news -- hearing now 100% and my results are back there also.

Only one general minor bugfix -- should appear to be more open sounding -- but not critical for update.

This version of the decoder is exactly everything that i was attempting 2 days ago, but working 100% correctly.  I had run tests (research) earlier, so I knew what to expect -- but this is more a result of more thorough measured results.   The new capabilities are as follows:
1) More complete anti-intermod code for the 30-3kHz range.  This processing goes far beyond anything that I have heard of.  The only audible difference is a small reduction in middle frequency graininess.  The improvement is very real, and this will eventually be the default way that the decoder is running.  Currently, this mode only functions at --ai=high or --ai=max.  The normal anti-intermod code works fine at --ai=med (the default mode.)  The anti-intermod code can be partially disabled at --ai=off, or fully disabled with --ai=none.   There is almost never a reason to disable the anti-intermod code except that the more advanced modes do take more CPU (high or max modes.)

2) Totally new anti-hash facility and a control for it.  I have found that many HF intense recordings (even mellow stuff like the Carpenters -- due to vocal sibilance) have a certain hash in the sound.   From what I can tell and looking at the spectrum, I believe that it is a 'beat' between the gain control on the various frequency ranges -- most likely between the slower 80-3k range (which actually extends beyond 10kHz to at least 13kHz) and the faster 3-9k/9k-20k ranges.  Since there is a feedback design in the encoder, it appears that the compressors are interacting in an ugly way.  ALL DolbyA recordings seem to have this problem depending on the density of the HF sound.  My processor has some code/logic to avoid the appearance of the grainy HF sound.
The anti-hash mode can be enabled/disabled by --ah=on or --ah=off, and is always disabled when the anti-intermod is fully disabled.  There should never (very, very seldom) be a need to disable the anti-hash code.

The new stuff is also explained in the DecoderA.pdf file.  The decoder has been run through lots of tests and spectral review -- no unexpected notches or dips anywhere :-).  There is a dynamic dip at 9kHz (1kHz wide, which corresponds to the attack rate of the DolbyA) which fills in when really needed -- and that is part of the anti-hash processing -- should be inaudible except for slightly cleaner sibliance.

Sorry again for the previous problems -- these items had been created a while ago -- but I had to previously revert a lot of code so that I could regain control after all of the previously broken changes.

This should make people happier than even before.

Peeling the onion -- the last improvements opened up more!.

Reply #121
Improvement:  detector code had some aliasing (almost the last sole source of obvious distortion) -- causing a hard edge, and also the attack was a little bit too slow (a counter-intuitive situation.)  Result after the readjustment -- simple/clearer voices without the hard edge.  None of the attack/decay general architecture was changed, but some minor improvements were made for better SIMPLE clarity while removing edge-buzz (hard edges.)  The signal attack is also followed more closely.  Without the edge-buzz the sound is cleaner -- but the sizzle might be missed.  You really don't want the sizzle -- it is full of all kinds of distortion uglies :-).  Probably could not hear this improvement before the last set of improvements decreased the other distortion sources

Background: After the last set of improvements, it opened up other possibilities of improving the audio.  The innate distortion of the internal operation of the code is practically NIL (massive amount of subtle math/filtering/etc).  This has made it possible for me to detect more (more and more subtle) problems elsewhere.  This one is a really good one -- one of my favorite subjects -- the 'level detector' and basic attack code became the biggest source of distortion again (after everything else has gotten better.)  This one was a real learning exercise for me -- and the side benefit is that the code much more closely follows the attack of the audio signal -- yet distorts impossibly little.  This has gotten to the point now where I cannot hear the 'edge' distortion from the detector at all (maybe my ears messed up again?  I don't think so -- this is really an improvement.)   You'll notice that the edge of voices (sometimes manifested) is now gone.  Karen' Carpenter's voice often has no real hard edge anymore!!!  This is getting DAMNED close to success.  ABBA still has chaos in the sound -- but the voices in the chaotic chorus are clearer.  Material like Dionne Warwick, Bread or other naturally mellow (but detailed) sound is more natural yet.  FEEL FREE TO BOOST THE TREBLE WHEN YOU WANT - the distortion is much diminished, and increasing treble isn't so painful anymore.

The features are the same -- I am not willy-nilly adding tunables anymore unless there is something that REALLY makes a difference.  The complexity of tunables and supporting so many code paths causes too many bugs -- so I am trying to keep things simpler -- maintaing quality as the singular goal (well, also consider CPU usage, which has gotten pretty bad lately.)

So, semi bad news regarding CPU usage -- the code barely runs in real-time on my Haswell (4 core, but only one is used by the program) 4770 processor.  Laptop ATOM/Silvermont processors run several times slower than realtime -- but I am sure that newer processors can run up to maybe 2-3times realtime.   I am NOT going to multi-thread the program -- I do have a really good multi-threading infrastructure, but I believe that there is some overdesign in the code, and I think that the next step will be to speed up the code without impacting quality.  Multi-threading only makes multi-platform support & debugging more trouble if simply speeding up the code can resolve the speed issue.

I am having a blast listening to my library with this processor -- simply clean sounding for the first time -- better than I remember from vinyl in many cases.


Usage hint for new users of the decoder.

Reply #122
Wanna give a quick usage hint -- if I was a new user, I might also have gotten tripped up by this...  That is, even though it will become obvious to follow this rule!!!

Never change the level of the input material unless you compensate for it.  For example -- if you need to decrease the input signal by 3dB (from the original file containing the recording), then you need to specify a gain (or provide gain) before or at the decoder.

So, if you have an input file, and then did a 'gain -3' in sox, then to compensate, an '--ingain=3.0' needs to be supplied on the decoder command line.  The gain between the music file and the decoder must be zero dB, or the threshold will never match.  (Of course, you could change the threshold to make the decoder work instead -- but then your signal levels will be wrong anyway.)  So -- it is best to keep everything gain 'neutral' and/or compensate by using the '--ingain=xx' switch on the decoder.

Just trying to help to avoid confusion!!

New update -- didn't expect this one!!!

Reply #123
Did a careful code inspection, and found a really stupid mistake.  The long term decay for the right channel was dependent upon the left channel!!!   Really silly, and I am surprised that I didnt' catch it by now because I do a careful listening test.  I guess that the longer term levels are generally fairly consistent, but now the code is more correct.

After finding this bug, I did a VERY careful code review to make sure that there were no other bugs like this.  The problem is that I named a lot of variables like this:  LINrmsinl and LINrmsinr, and some of the math results in fairly long lines of code.  There were two mistakes, one was simply in a normal place, and the other was at the end of a line.

This version should GENERALLY sound similar -- I think that there are a few minor improvements, but additionally this long term decay error/miscalculation is fixed!!!

Found one song that required a decoder patch

Reply #124
One of the decoder calculations didn't go deep enough, so I added some depth so that it would handle certain kinds of long duration beats (yep -- that has to be one of the calculations to emulate the feedback to feedforward calculations for the decay time!!!)  I cannot implement a forever set of calculations that have very diminishing returns -- I very slightly underestimated what was needed for the very worst case.

The difference between this one and the previous is pretty small -- both are good.  This one is SLIGHTLY better.  Whenever I find a problem, I want to distribute it ASAP -- since there were no downloads on the previous, maybe this is in time to avoid wasting anyones time with the previous (again -- it was good also)!!!

The only other difference is that I inlined more of the functions so as to get a percent or two faster runtime.


SimplePortal 1.0.0 RC1 © 2008-2018