Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR) (Read 15343 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #25
LossyWAVinsaneextremehighstandardeconomicportableextraportable
599526459428397348312
FSLAC-8-7-6-5-4-3-2
900863802731649565483

Thanks for the detailed comparison, Markuza97! Here's my final FSLAC 1.3.4 version, along with the modified source file.
This includes a minor last change to preset -2 such that the resulting average bit-rate should end up around 450 kbps (instead of 483 kbps), which makes that preset -2 comparable to LossyWAV's high preset.

Note that this final version can be identified by the displayed copyright year 2022 when calling the executable (btw, the FLAC developers should update the copyright year as well) and by the encoding tag "semilossy libFLAC 1.3.4 20220430" in the generated FLAC files.

Note to myself: average bit-rate progression = 350 + cumsum(110:-10:50)

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #26
I wanted to do something similar but codec agnostic, so it could be used by FLAC/WAVPACK/TTA/TAK/SHN/etc

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #27
I wanted to do something similar but codec agnostic, so it could be used by FLAC/WAVPACK/TTA/TAK/SHN/etc
Would be fascinating then, to run a corpus through say, "max 900" through N codecs and then see how much smaller it becomes when the processor has to cave in and reduce according to the worst codec for each block. That could give an idea of how much the codecs differ not only per genre/album/song, but fine-grained.

There are at least five lossless codecs - FLAC/ALAC/WavPack/TAK and MPEG-4 ALS - which can handle a a block size of exactly 4096 (stereo) samples. 

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #28
I think the 352 kbps option could be left as "preset -1", since 480 kbps is still high if we consider this codec as semi-lossless. It would be great if you consider creating preset -1.

The encoders I made with the old (352 kbps) and the new -2 preset (480 kbps) sound great, both are or very close to transparency.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #29
This FSLAC is VERY interesting.
I think it deserves further testing on few critical samples and general transcoding to see where it stands against WavPack lossy and LossyFLAC perhaps. I made few fast encoding/transcoding tests and it works fine (I've used final version FSLAC 134_450k.exe).
All things considered, this is excellent stuff. :)
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #30
Thanks. Since I fear that not everyone has fully understood the purpose of this project, let me clarify two aspects.

1. "Very close to transparency", as Verónica mentioned, is not the goal of the FSLAC encoder. Complete transparency, with a certain safety margin provided by the higher presets, is. If you're looking for a lossy encoder around 320-400 kbps (stereo, or half that for mono), please use a proper lossy codec. IgorC has blind-tested some recent contenders here in 2020, and some of these performed very well already around 192 kbps stereo (note that exhale has improved to a score of around 4.9 as well since then). Feel free to use the FSLAC binary in this previous post of mine to get down to 330 kbps or so for testing, but I highly recommend not to use that version anymore.

2. True VBR encoding, as LossyWAV does, is also not the intention of the FSLAC encoder. Highly constrained VBR (CVBR) encoding, with a strict limit on the instantaneous bit-rate, is. Please take the attached short lossless audio file as an example. When I encode this with, say, FSLAC preset -3, I get a bit-rate very close to the average bit-rate on a larger CDDA waveform collection (around 560 kbps according to Markuza97's post above). Encoding the same short file with FLAC -8 after processing by LossyWAV's preset portable, however, leads to a bit-rate of 800 kbps or so, which is more than twice as much as LossyWAVs average bit-rate on the larger collection at that preset (around 350 kbps according to Markuza97).

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #31
@C.R.Helmrich
So, all FSLAC presets (2-8) should be completely transparent?
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #32
Yes, except maybe on that triangle-8.lfac sample. If you can reliably ABX other samples, please report them here along with links to the original lossless audio.

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #33
I've tried to ABX this sample:
https://hydrogenaud.io/index.php?topic=120193.0

Original FLAC vs FSLAC preset -3

foo_abx 2.0.6c report
foobar2000 v1.6.5
2022-05-03 17:53:21

File A: Jari Aalto - Electronica (Sample).flac
SHA1: fa044d2f4662f4689707666fe971d64d4882093b
File B: Jari Aalto - Electronica (Sample)-FSLAC-3.flac
SHA1: 809cf0538682fb82a1cf0382ca288fe7f31edf06

Output:
Default : Primary Sound Driver
Crossfading: NO

17:53:21 : Test started.
17:54:53 : 01/01
17:55:23 : 02/02
17:55:31 : 03/03
17:55:39 : 04/04
17:55:47 : 05/05
17:55:55 : 06/06
17:56:02 : 07/07
17:56:09 : 08/08
17:56:17 : 09/09
17:56:24 : 10/10
17:56:24 : Test finished.

 ----------
Total: 10/10
p-value: 0.001 (0.1%)

 -- signature --
d271dae3ad68334190cdc907198dddd02758262f

Difference is very, very subtle. A tiny noise appears at the second drum hit.

FSLAC level 4

foo_abx 2.0.6c report
foobar2000 v1.6.5
2022-05-03 18:13:39

File A: Jari Aalto - Electronica (Sample).flac
SHA1: fa044d2f4662f4689707666fe971d64d4882093b
File B: Jari Aalto - Electronica (Sample)-FSLAC-4.flac
SHA1: fdd1c623d111843d58a2e1be82b096809a1b215c

Output:
Default : Primary Sound Driver
Crossfading: NO

18:13:39 : Test started.
18:14:53 : 00/01
18:15:10 : 01/02
18:15:35 : 02/03
18:15:39 : 03/04
18:15:46 : 04/05
18:15:51 : 05/06
18:16:11 : 06/07
18:16:17 : 07/08
18:16:35 : 07/09
18:16:59 : 08/10
18:16:59 : Test finished.

 ----------
Total: 8/10

Difference is very, very subtle. Super tiny noise at second drum hit. VERY hard to hear.


Difference disappears at FSLAC level 5.

I hope this helps. ;)
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #34
Thanks, it does help, a lot! Can you please ABX the attached version of FSLAC (preset -3) again and tell me whether things have improved?

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #35
Original FLAC vs Modified FSLAC level 3

foo_abx 2.0.6c report
foobar2000 v1.6.5
2022-05-04 17:40:45

File A: Jari Aalto - Electronica (Sample).flac
SHA1: fa044d2f4662f4689707666fe971d64d4882093b
File B: Aalto_sample_fslac-3_test.flac
SHA1: cf285c82c6761aee427f8c67c94f99644769d086

Output:
Default : Primary Sound Driver
Crossfading: NO

17:40:45 : Test started.
17:42:02 : 01/01
17:42:06 : 02/02
17:42:17 : 03/03
17:42:25 : 04/04
17:42:41 : 05/05
17:42:55 : 06/06
17:43:47 : 07/07
17:44:00 : 07/08
17:44:16 : 08/09
17:44:27 : 09/10
17:44:27 : Test finished.

 ----------
Total: 9/10
p-value: 0.0107 (1.07%)

 -- signature --
300260155009022e7453a206635489d7a1a05ab1


Definitely better. Difference is now very, very hard to hear.
I suppose this modification could be implemented in new FSLAC version?


lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #36
Yes, though I don't know when yet, I'm busy with other things and this requires more thorough testing. So please don't wait for it.

I wanted to do something similar but codec agnostic, so it could be used by FLAC/WAVPACK/TTA/TAK/SHN/etc
You can always transcode losslessly from FSLAC to another lossless codec. Or even to, say, FSLAC at preset -8, I do this sometimes with my FSLAC -3 encodings for that extra 1% of compression efficiency.

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #37
I have a corpus of 71 files non-classical CDDA tracks "arbitrarily picked by successive MD5 sums", and I let fb2k generate 10 seconds preview clips of each.  Bias: early in each track.

Some observations:
* In fb2k it identifies itself as "semilossy" rather than "semilossless", which should satisfy those of us who think that "semivirgin" is sounds like bad humour and "semislut" sounds like someone reasonable (also with bad humour).
* Decompressing sometimes yields MD5 mismatch. -8: 2 files. -7: 6 files. -6: 4 files. -5: 5 files    
* Recompressing a 1.3.4 semilossy losslessly by my 1.3.4 FLAC build yields slightly smaller files at -8 (overall change only 1 kbit/s) while for -6 I observe 0.992 to 1.009.
* Successive semilossy processings reduces the bitrate. That is in line with the previous comment? Numbers: 
927 kbit/s for -8 lossless
891 kbit/s for -8 semilossy (57 out of 71 clips differ to lossless, highest: -65.20 dBTP)
882 kbit/s for another -8 semilossy (51 out of 71 clips differ to the previously generated semilossy files)
881 kbit/s for another -8 semilossy (18 out of 71 clips differ to the previously generated semilossy files)
881 kbit again (2 differ)

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #38
Yes, though I don't know when yet, I'm busy with other things and this requires more thorough testing. So please don't wait for it.

I wanted to do something similar but codec agnostic, so it could be used by FLAC/WAVPACK/TTA/TAK/SHN/etc
You can always transcode losslessly from FSLAC to another lossless codec.

Turns out not to be worth it.  Same corpus as previous comment, compresses losslessly to 927 kbit/s using FLAC 1.3.4 at -8:

Semilossy bitrates then:
-8: 891, 57 clips differ out of 71
-7: 847, 64 clips differ
-6: 787, 67 clips differ
-5: 709, all clips differ.

Recompressed those using a few lossless codecs.  Since we are doing this to save bitrate, I wanted to take one step up from the default (except ALAC and TTA), but I don't want to be bothered with the slowest. Picked one setting per codec (edit: that became more!) as follows:

WavPack -hx.  Edit: just after posting I thought that hm, maybe the block size should be comparable to FLAC? So the second figure is -hx --blocksize=4096 , and that has more benefit from lossiness but still not the same as FLAC
919 resp 926 lossless
910 resp 906 compressing -8
884 resp 869 compressing -7
842 resp 815 compressing -6
787 resp 749 compressing -5



TAK -p3; edit: with enforced max 4096 here too
902 resp 904 lossless
885 resp 882 compressing -8
853 resp 844 compressing -7
803 resp. 789 compressing -6
739 resp. 723 compressing -5 (had to try -p4m: 745 vs 720).



Monkey's High: didn't expect anything since it doesn't support wasted bits.  All 900 except ... ape-ing the -5 yields 901   
TTA: As Monkey's, only 26 kbit/s higher.
ALAC: 945 all over.

But then comes the surprise:
OptimFROG --preset 4:  891 no matter what! That was such a surprise that I fiddled a little bit more around and managed to get oh-so-slight improvements. But the frog does not see the point of this.

So gains from lossiness are biggest with FLAC (that uses blocksize 4096), then TAK with max blocksize 4096 as good as tied to WavPack with blocksize 4096, then TAK, then WavPack and approximately zero for the rest.
Not anymore any size reason to migrate from FLAC to TAK, for example.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #39
Yes, though I don't know when yet, I'm busy with other things and this requires more thorough testing. So please don't wait for it.

Chris

No worries... Current version is already very good. :)
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #40
...
So gains from lossiness are biggest with FLAC (that uses blocksize 4096), then TAK with max blocksize 4096 as good as tied to WavPack with blocksize 4096, then TAK, then WavPack and approximately zero for the rest.
Not anymore any size reason to migrate from FLAC to TAK, for example.
Thanks very much for the thorough evaluation, Porcus! Interesting and somewhat surprising results. Maybe running F(S)LAC itself with a smaller block size would change this a bit?

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #41
FSLAC is an interesting tool. Thanks Christian :)

I have a small question. I played a bit with fslac 1.34, and it seems to handle 24 bit files without obvious issues.  My work is so far restricted to a tiny bitrate table. A collection of 12 files at 24/96. FLAC bitrate is 2674 kbps ; FSLAC bitrate varies from 700 kbps (preset 2) to 2109 kbps (preset 8).

According to http://www.ecodis.de/audio/fslac/issues.html :
Quote
The following is a list of operational issues known to the author as of December 31, 2016, which are likely to occur when running the 32-bit Windows executable fslac.exe  […]
.
Input bit-depths greater than 20 bit per sample are not supported.

The reason is a potential accumulator overflow in the constrained variable bit-rate (CVBR) calculation, which may lead to incorrect results in terms of instantaneous output bit-rate. I am not going to modify the code to correct this issue. If you think you need 24 bit per sample, FSLAC is probably not for you, anyway, and I recommend you apply conventional, completely lossless compression, e.g. using FLAC.

Was the issue recently corrected? If not, what kind of (wrong) behaviour is expected when using 24 bit files with fslac?

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #42
Nice hearing from you, guruboolez! :) It's been a while since I wrote that list of FSLAC issues, but I think all issues except, maybe, for the 5. Unicode thing are still present in the above updated version. If memory serves, the issue with high bit-depths is that different functions are used in the FLAC library, and I was too lazy to re-implement my FSLAC changes there. But maybe the FLAC source code has changed since then and it works now ;) The FSLAC bit-rates you report seem to be in an expected range.

Thanks for testing this!

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #43
Thank you Christian for your clarification. In other words, issues may appear with 24 bit fslac so usage is not recommanded. Is it correct?

Anyway if you need help on tests, bitrate, anything… don't hesitate :)

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #44
If memory serves, the issue with high bit-depths is that different functions are used in the FLAC library, and I was too lazy to re-implement my FSLAC changes there

Those different codepaths are in lpc.c and fixed.c which you didn't touch, the selection is in stream_encoder.c. I see there under an #ifdef ECODIS_CVBR_MODE
Code: [Select]
if (subframe_bps + 4 + FLAC__bitmath_ilog2((quarter_blocksize-FLAC__MAX_FIXED_ORDER)|1) <= 32)
    fixed_order = encoder->private_->local_fixed_compute_best_predictor(integer_signal+FLAC__MAX_FIXED_ORDER, quarter_blocksize-FLAC__MAX_FIXED_ORDER, fixed_residual_bits_per_sample);
else
    fixed_order = encoder->private_->local_fixed_compute_best_predictor_wide(integer_signal+FLAC__MAX_FIXED_ORDER, quarter_blocksize-FLAC__MAX_FIXED_ORDER, fixed_residual_bits_per_sample);

I see that the above bit of code is also applied in what I suspect is the code detecting what peak to expect (in process_subframes_). Selection in evaluate_lpc_subframe_ doesn't seem to be touched by your changes either. So either it has been fixed, or the problem is a little more subtle?
Music: sounds arranged such that they construct feelings.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #45
Jup, it's more subtle. Re-reading my Issue list linked to above, it seems to be a possible int32 or int64 overflow in some of my newly added average-calculation. So yes, due to this, FSLAC encoding of 24-bit audio files, especially noisy or loud ones, is not recommended.

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #46
@C.R.Helmrich Could you update it with FLAC 1.4.2 please?

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #47
@C.R.Helmrich
Is there any update or progress regarding FSLAC?

Thanks.
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #48
Nope, no updates, this has really been (or was intended as) only a small side project. I've actually experimented with higher-rate exhale encoding, for stereo CD audio around 300 kbps (maximum 400 kbps) - see this post of mine -  which is far more efficient than using FLAC for lossy encoding purposes.

Chris
If I don't reply to your reply, it means I agree with you.

Re: FSLAC: A FLAC Backward-Compatible Free Semi-Lossless Audio Coder (NMR)

Reply #49
Nope, no updates, this has really been (or was intended as) only a small side project. I've actually experimented with higher-rate exhale encoding, for stereo CD audio around 300 kbps (maximum 400 kbps) - see this post of mine -  which is far more efficient than using FLAC for lossy encoding purposes.

Chris

Thanks a lot for clarification Chris. :)
I have 3 questions:
1. Regarding exhale: Can I encode cd audio with 300-400k bitrate already or this feature will be implemented in future versions?
2. If it's available already, which command line switch should be used?
3. Regarding FSLAC (as I really like it): Can I rely on it as it is regarding stability? Is it considered safe so I don't have to worry that my
encodes could end up corrupted or with some hidden erros - stuff like that? (in my limited testing so far it was completely stable).

Thanks a lot.
Best regards.
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)