Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: HALAC (High Availability Lossless Audio Compression) (Read 109666 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #225
Thank you very much ktf for the good information you have given. I did not think that there could be such exceptional cases in stereo data. If this is the case, it can really be better efficiency in these special cases.  I can come up with a quick solution for this in the next releases.
Well, I don't think it is really exceptional at all. The reason transforming L+R to M+S works very well most of the time is because most music is checked for mono compatibility, this is important for broadcasting to mono radio receivers. Also, excessive phase differences would cause problems with skipping on vinyl records. While vinyl hasn't been dominant for years, old techniques tend to stick around.

Also, you can have quite nice stereo while remaining mono compatible. But truly great stereo usually needs phase differences as well. As you have noticed, one bit of audio HALAC underperformed was Armonia Di Flauti. Guess what: great, true stereo (I really enjoy it) but not achieved by only varying amplitude, but both amplitude and phase. HALAC isn't the only one though, as far as I know, WavPack does this as well: always converting L+R to M+S.
Music: sounds arranged such that they construct feelings.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #226
I found some time and did a quick test. I was a bit lazy to write the detailed results. So I'll summarize briefly.
The processor, memory and ssd I used for the test is not very fast. I did this to see compatibility with legacy systems. However, when I use a newer processor, HALAC's decode speed increases even more. And FLAC -5 has a faster decode time than FLAC -0.

16 bit, 48000 bps, stereo, 4 CD
Total size: 2,462,609,998 bytes

HALAC -normal13.268s17.114s1,727,197,297 bytes
FLAC -534.249s18.968s1,729,332,719 bytes

Re: HALAC (High Availability Lossless Audio Compression)

Reply #227
Also, you can have quite nice stereo while remaining mono compatible. But truly great stereo usually needs phase differences as well. As you have noticed, one bit of audio HALAC underperformed was Armonia Di Flauti. Guess what: great, true stereo (I really enjoy it) but not achieved by only varying amplitude, but both amplitude and phase. HALAC isn't the only one though, as far as I know, WavPack does this as well: always converting L+R to M+S.
I thought “Armonia Di Flauti” was in mono, like “Shakespeare Lectures”, but it's not. There would be a real benefit in addressing such situations. I will take care of it separately.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #228
I found some time and did a quick test. I was a bit lazy to write the detailed results. So I'll summarize briefly.
The processor, memory and ssd I used for the test is not very fast. I did this to see compatibility with legacy systems. However, when I use a newer processor, HALAC's decode speed increases even more. And FLAC -5 has a faster decode time than FLAC -0.

16 bit, 48000 bps, stereo, 4 CD
Total size: 2,462,609,998 bytes

HALAC -normal13.268s17.114s1,727,197,297 bytes
FLAC -534.249s18.968s1,729,332,719 bytes
Thank you for the test. Yes, there may be a performance increase on newer systems.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #229
How "bad" it would be to stick to L&R as only choice? I don't know. Too lazy to test, I would need to use other tools to convert first.
I meant: how bad it would be to stick to Mid&Side as only choice.

So well, ... tested it: https://hydrogenaud.io/index.php/topic,121770.msg1055864.html#msg1055864

Re: HALAC (High Availability Lossless Audio Compression)

Reply #230
I have a question about HALAC's Rice Coder. Let all values ​​in a block of audio be 0 except one. The value that is different is 1000. And let the calculated Rice parameter be 0.
How many bits do you use to encode the value 1000 for k=0?

Re: HALAC (High Availability Lossless Audio Compression)

Reply #231
I have a question about HALAC's Rice Coder. Let all values ​​in a block of audio be 0 except one. The value that is different is 1000. And let the calculated Rice parameter be 0.
How many bits do you use to encode the value 1000 for k=0?
13 + 1 + 13 = 27 bit
Normally a unary encoding of 1001 bits is required. These situations are very rare indeed. HALAC has a method to prevent such exceptions. That is, it interferes with bit sets that are too long.  But in this case there is branching. And this has a negative impact on speed, especially in the decode phase.
I've done some tests and found that even without such a precaution, it doesn't really affect the compression ratio. I might revisit this.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #232
HALAC +
I am publishing this trial version, especially for those who want to see a HALAC capable of compressing a little more. Actually, I certainly didn't want to do something like this. Because HALAC was not developed for this. However, I wanted to clarify that better compression rates can be obtained without compromising more than speed.
This work, which I call HALAC+, is not an official version and is only a experiment for now. The compression ratio is very close to FLAC -8. However, the Encoding speed is least 4x faster. I didn't hardly deal with the Decoder optimization. So it's a little slow for now. But new processors can be faster.

Basically, I tried Adaptive Rice Coding. And a simple error correction. Block costs are a bit more. I think I can reduce them. Of course, this is not perfect and better can be done.
In this version, "-fast", "-ufast" parameter and lossyWav support for the time being disabled.

Code: [Select]
i7 3770K, 16gb, 240 gb. Single thread results
Busta Rhymes (829,962,880)
HALAC +   7.852  9.226  558,783,643
FLAC -8  33.128  7.113  558,662,099

Sean Paul (525,065,800)
HALAC +   5.259  5.952  373,657,717
FLAC -8  21.133  4.710  373,252,115

Globular (802,063,984)
HALAC +   7.342  8.699  473,432,182
FLAC -8  31.297  6.298  471,599,215

Sibel Can (504,822,048)
HALAC +   4.895  5.641  356,662,028
FLAC -8  20.455  4.489  356,997,976

Gubbology (671,670,372)
HALAC +   5.976  7.038  361,180,016
FLAC -8  26.233  5.244  360,222,227




Re: HALAC (High Availability Lossless Audio Compression)

Reply #236
Quickly tested only "track 02" from each of my 38 CDs. Not quite at FLAC -8:

1097724363 for flac -2b4096 --no-padding, using only fixed predictors
1042285340 for HALAC 0.38
1039976110 for flac --no-padding (default -5 but without the padding block for tags)
1039947667 for HALAC+. It is classical music where it cannot completely keep up with flac at default, and the rest that keeps it ahead. But classical music is a bit overweighted (more seconds) in the corpus.
1038935920 for TAK -p0
1038826778 for flac -l9 --no-padding (-l9 allows one order higher predictors than default -5)
1033674327 for flac -8 --no-padding
1009709572 for TAK at default -p2

Re: HALAC (High Availability Lossless Audio Compression)

Reply #237
Thank you, Porcus.
As we keep the test set wider, HALAC+ can probably give slightly better results. Even so, the differences between the results are at the level of 0.1% - 0.2%. So in this case, I think there is no need to reduce the processing speed to get a very small compression. However, I think I can make HALAC+ a little better in terms of speed/compression ratio.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #238
HALAC +
I am publishing this trial version, especially for those who want to see a HALAC capable of compressing a little more. Actually, I certainly didn't want to do something like this. Because HALAC was not developed for this. However, I wanted to clarify that better compression rates can be obtained without compromising more than speed.
This work, which I call HALAC+, is not an official version and is only a experiment for now. The compression ratio is very close to FLAC -8. However, the Encoding speed is least 4x faster. I didn't hardly deal with the Decoder optimization. So it's a little slow for now. But new processors can be faster.
According to my results, Halac+ is on average equivalent to Flac Level 8 as mentioned. Sometimes a little below, sometimes a little above. However, it compresses more than 4 times faster than the fastest thing known (Flac). And it is not even fully optimized.
But I think there is no need for such a thing. I can guess that Halac can offer better compression if desired, but it is not worth such a loss of speed for a small gain. Absolutely!

Re: HALAC (High Availability Lossless Audio Compression)

Reply #239
253 full albums to feed the comparison :)

  • HALAC 0.29 normal : 867 kbps on average
  • HALAC 0.38 normal: 851 kbps on average (-16 kbps compared to 0.29)
  • HALAC+ normal: 848 kbps (-3 kbps compared to HALAC 0.38)
  • HALAC 0.29 fast: 909 kbps on average
  • HALAC 0.38 fast: 869 kbps on average (-40 kbps compared to 0.29!)
For comparison, FLAC 1.43 -8 end with 841 kbps (-7kbps compared to HALAC+)


Full and detailed results on my table:
https://docs.google.com/spreadsheets/d/1V-OwngNspELbkoRp-DQ34wMj2g7g-VMtvqaCqpqMAoU/edit?usp=sharing
(go to the FULL table tab, and all HALAC values are at the extreme end of the table)

On summary:
HALAC+ is on average 1 to 4 kbps smaller than HALAC 0.38 depending on musical genre
HALAC+ is bigger than FLAC -8 on all genres except AUDIOBOOKS corpus (-1 kbps). Biggest difference found on the ELECTRONIC MUSIC corpus (+13 kbps for HALAC+ compared to FLAC).

I can't comment speed encoding (bottlenecked by external drives/USB)

Wavpack Hybrid -c4hx6

Re: HALAC (High Availability Lossless Audio Compression)

Reply #240
According to my results, Halac+ is on average equivalent to Flac Level 8 as mentioned. Sometimes a little below, sometimes a little above. However, it compresses more than 4 times faster than the fastest thing known (Flac). And it is not even fully optimized.
But I think there is no need for such a thing. I can guess that Halac can offer better compression if desired, but it is not worth such a loss of speed for a small gain. Absolutely!
Yes, HALAC is a speed-oriented codec. I wanted to show something like this because I was asked if you can get better compression by sacrificing some speed. But there may be other things in the next process.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #241
253 full albums to feed the comparison :)

  • HALAC 0.29 normal : 867 kbps on average
  • HALAC 0.38 normal: 851 kbps on average (-16 kbps compared to 0.29)
  • HALAC+ normal: 848 kbps (-3 kbps compared to HALAC 0.38)
  • HALAC 0.29 fast: 909 kbps on average
  • HALAC 0.38 fast: 869 kbps on average (-40 kbps compared to 0.29!)
For comparison, FLAC 1.43 -8 end with 841 kbps (-7kbps compared to HALAC+)


Full and detailed results on my table:
https://docs.google.com/spreadsheets/d/1V-OwngNspELbkoRp-DQ34wMj2g7g-VMtvqaCqpqMAoU/edit?usp=sharing
(go to the FULL table tab, and all HALAC values are at the extreme end of the table)

On summary:
HALAC+ is on average 1 to 4 kbps smaller than HALAC 0.38 depending on musical genre
HALAC+ is bigger than FLAC -8 on all genres except AUDIOBOOKS corpus (-1 kbps). Biggest difference found on the ELECTRONIC MUSIC corpus (+13 kbps for HALAC+ compared to FLAC).

I can't comment speed encoding (bottlenecked by external drives/USB)
Thank you so much for tests, guruboolez.
HALAC 0.3.8 -fast mode seems to have improved really well. And this mode is really good in terms of compression ratio/speed.

Re: HALAC (High Availability Lossless Audio Compression)

Reply #242
I gave the Linux executables a quick try on a couple large files...

FLAC 1.5.0 vs HALAC 0.3.9 (from post #235, I renamed the executables to halac_enc & halac_dec for readability for the test)

file_01.wav: 16-bit 2-ch, 1h 43m, 1098612188 bytes    (Nine Inch Nails - The Fragile)
file_02.wav: 16-bit 2-ch, 2h 11m, 1391153948 bytes    (Fear Factory - Demanufacture+Remanufacture)

I re-tested file_01 a few times.  I get the same corruption error each time.   The options -fast and -ufast also don't work for me.  No error messages, but no output either.

Encoding...
Code: [Select]
time ./flac --silent -5 file_01.wav

real 0m6.782s
user 0m6.114s
sys 0m0.668s
Code: [Select]
time ./halac_enc file_01.wav file_01.halac -normal

real 0m4.965s
user 0m4.600s
sys 0m0.365s

Code: [Select]
time ./flac --silent -5 file_02.wav

real 0m8.712s
user 0m7.816s
sys 0m0.896s
Code: [Select]
time ./halac_enc file_02.wav file_02.halac -normal

real 0m6.317s
user 0m5.833s
sys 0m0.484s

Code: [Select]
stat -c "%s %n" -- file_*
662438138 file_01.flac
662341661 file_01.halac
973741394 file_02.flac
973935321 file_02.halac

Decoding...
Code: [Select]
time ./flac --silent -d file_01.flac

real 0m5.597s
user 0m4.872s
sys 0m0.725s
Code: [Select]
time ./halac_dec file_01.halac file_01.wav
The decoded file has a corruption in the frame 1373! (around byte 719847424)

real 0m6.887s
user 0m6.226s
sys 0m0.661s

Code: [Select]
time ./flac --silent -d file_02.flac

real 0m7.112s
user 0m6.187s
sys 0m0.924s
Code: [Select]
time ./halac_dec file_02.halac file_02.wav

real 0m9.118s
user 0m8.330s
sys 0m0.788s

Re: HALAC (High Availability Lossless Audio Compression)

Reply #243
This work, which I call HALAC+, is not an official version and is only a experiment for now. The compression ratio is very close to FLAC -8. However, the Encoding speed is least 4x faster. I didn't hardly deal with the Decoder optimization. So it's a little slow for now. But new processors can be faster.

Basically, I tried Adaptive Rice Coding. And a simple error correction. Block costs are a bit more. I think I can reduce them. Of course, this is not perfect and better can be done.
In this version, "-fast", "-ufast" parameter and lossyWav support for the time being disabled.
Thanks for the test replica9000.
As I mentioned in the previous post, the version 0.3.9 is still a draft/experimental work. It is a slower and limited version to keep the compression ratio a little high. I know the exception in Decoder. A small overflow that is overlooked during the error correction stage. Since it was very rare, I was able to notice much later than sharing. However, I could not update because it was already experimental.

In addition, I cannot say that I do excess speed optimization for Linux. The compilers I use are not up to date. I'm actually working on Windows. But you can do your tests on Linux on 0.3.8 for now.