Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: HALAC (High Availability Lossless Audio Compression) (Read 55241 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #200 –
Quote from: Porcus on
Quote from: genuine on
Can't the crazy Rice encoder and decoder that you used in HALAC also be used in FLAC?
That is kinda what I suggested above. A credit for speeding up FLAC by xx percent, would make more waves than a proof-of-concept codec.
Those who pay attention know that FLAC is fast already and has been honed well over > two decades.

But ... developer's choice.
I really don't know how the things you're describing are going to happen and what's going to happen in the end. I don't know what a "credit" is and what it is useful for.
However, I think what I understand in a nutshell is to leave HALAC and support FLAC.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #201 –
What changes you did to LPC part?

Re: HALAC (High Availability Lossless Audio Compression)

Reply #202 –
Quote from: Hakan Abbas on
I don't know what a "credit" is and what it is useful for.
Credit = acknowledging that some thing was done by an actual human being that exists
That's exactly what a "credit" is and exactly what it's useful for.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #203 –
Quote from: mycroft on
What changes you did to LPC part?
It is important to feed the linear estimator with clean data. However, I noticed that I was including a few unnecessary data in the calculation from the beginning. Because the squaring of the differences can change the sum value quite a lot. And this means drawing the wrong function.
In previous versions, sometimes the “-fast” mode was even better than the “-normal” mode. And the simpler audio data was not compressed enough. Now we are much more robust.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #204 –
Quote from: Lampion on
Quote from: Hakan Abbas on
I don't know what a "credit" is and what it is useful for.
Credit = acknowledging that some thing was done by an actual human being that exists
That's exactly what a "credit" is and exactly what it's useful for.
Hmm. Thank you very much for the information.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #205 –
Yeah, it means the changelog will say who did it.
That will get your work better known, than impressing a web forum will.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #206 –
Speaking of which, impress the FLAC team and existing FLAC users into making an actual version of FLAC with HALAC features and concepts instead of impressing random people into a new HALAC version entirely.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #207 –
Quote from: Hakan Abbas on
Thank you for your good thoughts. Since the first version of HALAC, an average of 2 percent compression has been obtained. Without compromising the speed. My guesses were in this direction. However, I think a little more improvement can be done without compromising speed.

I can't say that I finished exactly my Rice Coder. In fact, it needs to work a little faster. Especially the Decoder part. Dealing with the Coder can sometimes be badly annoying.

I don't know the Rice Coder used by FLAC. However, they should be quite different structurally with mine. Entropy coder change can cause back compatibility problems. We don't know that. My own Rice coder is now embedded in HALAC. It needs to be abstracted in a good way. I mean, I've never thought of anything like this before.
I'm taking my question back. Because the issue went to another side. HALAC != FLAC

FLAC has already been among us for years. And years later we see a new lossless audio codec. It is not easy to develop a real codec and those who do not develop it cannot know. But it is more difficult to develop a successful codec. I see this work very successful.
Turning such a work into something like "credit" means clearly destroying it. I think this is not a good idea at all. Let us continue to develop. He said that he would already have problems with FLAC. Completely different structures and approaches...

Re: HALAC (High Availability Lossless Audio Compression)

Reply #208 –
Quote from: Hakan Abbas on
HALAC version 0.3.8 performs a more successful linear prediction process. [...]
Very impressive. I've ran a short variant of my comparison script (on 8 different tracks) and the results are attached.

Clearly, there is no codec that comes close to HALACs encoding speed. Moreover, this test is single-threaded only, and the way multithreading is implemented seems way more efficient than other codecs.
Music: sounds arranged such that they construct feelings.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #209 –
actually nvm just keep working on HALAC

Re: HALAC (High Availability Lossless Audio Compression)

Reply #210 –
Quote from: ktf on
Quote from: Hakan Abbas on
HALAC version 0.3.8 performs a more successful linear prediction process. [...]
Very impressive. I've ran a short variant of my comparison script (on 8 different tracks) and the results are attached.

Clearly, there is no codec that comes close to HALACs encoding speed. Moreover, this test is single-threaded only, and the way multithreading is implemented seems way more efficient than other codecs.
Thank you very much for the test and your valuable feedback. I am glad that you confirm my results.
Are there “Shakespeare Lectures” or “Armonia di Flauti” among the 8 tracks you used? Because I think these are actually mono. In the previous tests, the correlation between the channels seemed to confuse things. I was going to edit this but I forgot. I will address this in the next version.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #212 –
Does HALAC use any predictions similar to TAK? Including advanced multi-channel correlation?

If I find recent papers about better predictions causing better compression ratios I gonna write another lossless audio codec :)

Re: HALAC (High Availability Lossless Audio Compression)

Reply #213 –
* It seems HALAC 0.3.8 does make use of stereo decorrelation, even in the -ufast option: "mono as stereo" files compress much better than with flac -0, which is dual mono.
* Does it still use blocksize 2048? I compared to that.

A quick test of compressions, not (yet) speeds, with my 38 CDs.

-ufast compresses slightly better than flac -0b2048, but the difference is all down to the "mono as stereo" files. Without them, HALAC would still compress better on the metal, but worse on the classical and "none of these" sections.
-fast was compared to flac -3b2048 which is also dual mono.  They would have been practically tied (52.83% vs 52.81) with HALAC number on the stereo-as-mono.  Also here, HALAC does better on metal and worse on classical.
-normal was compared to flac -5b2048 (that is, default except half the block size).  HALAC loses narrowly in all three genres, biggest difference was 0.21 percentage points for the classical section.

All FLAC were run with --no-padding

Re: HALAC (High Availability Lossless Audio Compression)

Reply #214 –
Tested encoding/decoding, both from file to NUL against FLAC in a a recent git build and checked roughly afterwards that it is faster than 1.4.2.
Compared to the results just posted by @ktf, HALAC outdoes FLAC at decoding in a much more pronounced way. The smaller differences at encoding could be because files are read from SSD, no RAM disk.

Corpus: my 38 CDs, one file per.

Encoding resp. decoding times in second, all are file-on-SSD to NUL:
40.4   57.2   encode/decode HALAC -ufast
49.3   97.3   encode/decode HALAC -fast
63.9   105    encode/decode HALAC normal
66.9   207    encode/decode FLAC -0b2048 -ss --no-md5
93.4   220    encode/decode FLAC -4b2048 -ss --no-md5

Here instead of doing flac at -3 and -5, I went for the middle -4 instead. It uses the -M switch that "only every now and then" re-evaluates the stereo decorrelation strategy.

Times are median of three runs, except FLAC decoding are best of two. Quite stable time consumption, in contrast to what happens if I try to write files.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #215 –
Quote from: mycroft on
Does HALAC use any predictions similar to TAK? Including advanced multi-channel correlation?

If I find recent papers about better predictions causing better compression ratios I gonna write another lossless audio codec :)
I don't know exactly how TAK works, I only have some preliminary knowledge about FLAC and SLAC. I want to proceed as much as possible without being bound and influenced by certain things. This way I can be more free.
I make normal use of the correlation between the channels, so there is no separate evaluation for each block. Because in normal stereo music the correlation between the channels remains the same in most cases throughout the music.
If we want to make something that works fast, we can't do many operations. For example, sorting, median, entropy, error correction, adaptive estimation, adaptive parameterisation, advanced/complex estimation.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #216 –
Quote from: Porcus on
* Does it still use blocksize 2048? I compared to that.
HALAC 0.3.8 uses 1536 samples per block in “-normal” mode. In “-fast” and “-ufast” modes it uses 2048 samples per block. However, there is actually no significant difference between them.

Quote from: Porcus on
Tested encoding/decoding, both from file to NUL against FLAC in a a recent git build and checked roughly afterwards that it is faster than 1.4.2.
Compared to the results just posted by @ktf, HALAC outdoes FLAC at decoding in a much more pronounced way. The smaller differences at encoding could be because files are read from SSD, no RAM disk.

Corpus: my 38 CDs, one file per.

Encoding resp. decoding times in second, all are file-on-SSD to NUL:
40.4   57.2   encode/decode HALAC -ufast
49.3   97.3   encode/decode HALAC -fast
63.9   105    encode/decode HALAC normal
66.9   207    encode/decode FLAC -0b2048 -ss --no-md5
93.4   220    encode/decode FLAC -4b2048 -ss --no-md5

Here instead of doing flac at -3 and -5, I went for the middle -4 instead. It uses the -M switch that "only every now and then" re-evaluates the stereo decorrelation strategy.

Times are median of three runs, except FLAC decoding are best of two. Quite stable time consumption, in contrast to what happens if I try to write files.
Thank you Porcus for the tests and the information. Yes, actually in my tests HALAC decode times are better. But we don't know exactly about ktf's system. But we can roughly say that it is faster than FLAC.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #217 –
Quote from: Hakan Abbas on
Are there “Shakespeare Lectures” or “Armonia di Flauti” among the 8 tracks you used? Because I think these are actually mono.
No, this is a different set of tracks. They are all plain stereo music.
Music: sounds arranged such that they construct feelings.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #218 –
Quote from: Hakan Abbas on
I make normal use of the correlation between the channels, so there is no separate evaluation for each block. Because in normal stereo music the correlation between the channels remains the same in most cases throughout the music.
So they are always transformed to average & difference?

The FLAC reference encoder will brute-force between dual mono and left+side, mid+side, right+side in five of the nine presets.  Among the four remaining, -0 and -3 encode dual mono, and in -1 and -4 it will make a selection and let it live on for a number of blocks before brute-forcing a block again.

@mycroft refers to how TAK selects multichannel decorrelation. Which has big effects: http://audiograaf.nl/losslesstest/revision%206/Average%20of%205.1%20surround%20sources.pdf
FLAC encodes independent channels (n-fold mono) except for stereo. IIRC, WavPack will default to pairing together FL & FR, pair together BL & BR, and SL & SR etc.

But as of now, HALAC is stereo-only?
 

Re: HALAC (High Availability Lossless Audio Compression)

Reply #219 –
Normally, in stereo (a simultaneous 2-channel recording of sound), the 2 channels of music should be related to each other. If there is no real relationship between the channels, we can't really call it stereo. So we are talking about 2 completely different channels. Most probably one channel is completely silent. Or a very special different situation. Of course there are such cases. But they are exceptions.

HALAC works in stereo by default right now. But as I said, I can also make it work in mono for completely independent channels.
In 5.1 or other multichannel audio, correlation can be exploited quite well. It's interesting that FLAC doesn't do that. As I mentioned earlier, if I ever start with multichannel audio data, I will need to build a fast channel analyzer.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #220 –
Quote from: Hakan Abbas on
Normally, in stereo (a simultaneous 2-channel recording of sound), the 2 channels of music should be related to each other.
Yes, but this relationship can take on various forms. There are live recording methods with two microphones in virtually the same spot but at an angle, then both channels are in phase but with a different amplitude. This is also common in overdubbing (most popular music). In some cases, panning is hard left or right, and a sound might be in one channel but not in the other.

You can also record stereo with only phase differences, which is not uncommon in classical music. Both channels are equal amplitude but with different phase. That is much more difficult to decorrelate.

A mix of both is also possible, is usually considered superior, and quite common for live recordings.
Music: sounds arranged such that they construct feelings.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #221 –
To give an idea from what the reference FLAC encoder does to my 38 CDs, brute-forcing the stereo decorrelation choice for each 4096 sample block:
Around 30 percent is encoded dual mono. But for the classical section, that increases to 46 percent. There are no big differences over encoding settings, as to whether variable LPC predictors are allowed or only fixed ones.

That's with FLAC's options, where it can choose not only between L&R and M(id)&S(ide), but also L&S and R&S (which, I can only guess, is motivated from the brute-forcing: if at first you calculate L, R, M, S, you get for virtually free to choose S + "smallest".)
So that gives a mild bias against dual mono, compared to if the choices were only L&R, M&S. Note, the "bias" from how the encoder defaults to dual mono for completely silent blocks (where all four choices would do equally), is less than 0.4 percent.
How "bad" it would be to stick to L&R as only choice? I don't know. Too lazy to test, I would need to use other tools to convert first.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #222 –
Quote from: ktf on
Quote from: Hakan Abbas on
Normally, in stereo (a simultaneous 2-channel recording of sound), the 2 channels of music should be related to each other.
Yes, but this relationship can take on various forms. There are live recording methods with two microphones in virtually the same spot but at an angle, then both channels are in phase but with a different amplitude. This is also common in overdubbing (most popular music). In some cases, panning is hard left or right, and a sound might be in one channel but not in the other.

You can also record stereo with only phase differences, which is not uncommon in classical music. Both channels are equal amplitude but with different phase. That is much more difficult to decorrelate.

A mix of both is also possible, is usually considered superior, and quite common for live recordings.
Thank you very much ktf for the good information you have given. I did not think that there could be such exceptional cases in stereo data. If this is the case, it can really be better efficiency in these special cases.  I can come up with a quick solution for this in the next releases.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #223 –
Quote from: Porcus on
How "bad" it would be to stick to L&R as only choice? I don't know. Too lazy to test, I would need to use other tools to convert first.
Code: [Select]
SQUEEZE CHART: 606,527,908 (100.00%)
HALAC L - R  : 367,531,670 ( 60.59%)
HALAC M - S  : 359,453,756 ( 59.26%)

BUSTA RHYMES:  829,962,880 (100.00%)
HALAC L - R :  597,675,972 ( 72.01%)
HALAC M - S :  561,170,930 ( 67.61%)

Re: HALAC (High Availability Lossless Audio Compression)

Reply #224 –
Yeah, but compared to a scenario where you can choose for each block.

If you look at the two fastest FLAC settings at http://audiograaf.nl/losslesstest/revision%206/Average%20of%20all%20hi-res%20sources.pdf and http://audiograaf.nl/losslesstest/revision%206/Average%20of%20CDDA%20sources.pdf , the difference is that -0 is dual mono, while -1 does "every now and then" check stereo decorrelation, and stick to that choice for a number of blocks before reconsidering. It improves a couple of percentage points for really cheap.
Then -2 only differs by checking all stereo decorrelation options for every block, and you see that is not at all equally cheap.

(-0/-1/-2 are not the fastest you can get FLAC, due to the block size. But "retuning" the presets isn't something you do without thorough testing.)