Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: TSAC: Very Low Bitrate Audio Compression (Read 7102 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Re: TSAC: Very Low Bitrate Audio Compression

Reply #25 –
Quote from: btc on
TSAC @ 8 kbps is comparable to Opus @ 24 kbps, maybe a bit more too. Great, I'll give you that. But that's it. TSAC has the same limitation as all other AI-based audiocodecs. It doesn't scale up with higher bitrate.  Nothing remotely close even to the modest HE-AAC@48 kbps, leave alone any higher bitrates/transparency levels.



Comparable how? I'd say TSAC @<any bitrate> is not comparable to <non-AI codec>@<any bitrate>. They are just too different for one to make such comparisons.

TSAC, especially at high bitrates, is very impressive actually. Better than any other codec I came across. Too bad it's not usable in practice, or even open-source (allowing experimentation, alternative models, ..etc).

Re: TSAC: Very Low Bitrate Audio Compression

Reply #26 –
Quote from: 2012 on
Comparable how? I'd say TSAC @<any bitrate> is not comparable to <non-AI codec>@<any bitrate>. They are just too different for one to make such comparisons.
Maybe by some double-blind listening tests? I am conducting the listening test of the TSAC, and comparison is possible.
TSAC at -q 12 sounds like a traditional codec at low bitrates, especially like Ogg Vorbis, with its sandpaper-like noise.

Re: TSAC: Very Low Bitrate Audio Compression

Reply #27 –
Quote from: 2012 on
@Kamedo2

How is the decoding speed when CUDA is utilized. Software decoding is actually slower than encoding on my workstation.

OK, I will test the decoding speed on RTX 3060.
Yes, CPU decoding is slower than the CPU encoding in my AMD Ryzen 7 5700X, too.

Re: TSAC: Very Low Bitrate Audio Compression

Reply #28 –
Quote from: 2012 on
Comparable how? I'd say TSAC @<any bitrate> is not comparable to <non-AI codec>@<any bitrate>. They are just too different for one to make such comparisons.
AI (LACE, NoLACE)  vs non-AI (Opus)
https://opus-codec.org/demo/opus-1.5/nolace_results.png
https://opus-codec.org/demo/opus-1.5/

AI (descript-audio-codec: TSAC based on it) vs non-AI (Opus)
https://github.com/descriptinc/descript-audio-codec/blob/main/assets/objective_comparisons.png

It's possible to compare any audio codec to another one, regardless of technology.

Re: TSAC: Very Low Bitrate Audio Compression

Reply #29 –
Interesting, but there isn't any "heavy" music (which is the most difficult to shrink) encode sample to evaluate...
Hybrid Multimedia Production Suite will be a platform-indipendent open source suite for advanced audio/video contents production.
Official git: https://forart.it/HyMPS/

Re: TSAC: Very Low Bitrate Audio Compression

Reply #30 –
Quote from: forart.eu on
Interesting, but there isn't any "heavy" music (which is the most difficult to shrink) encode sample to evaluate...
Well, download it and try for yourself. I've tried it with "Gimme hope Joanna" because reasons :)
TAPE LOADING ERROR

Re: TSAC: Very Low Bitrate Audio Compression

Reply #31 –
Quote from: btc on
AI (descript-audio-codec: TSAC based on it) vs non-AI (Opus)
https://github.com/descriptinc/descript-audio-codec/blob/main/assets/objective_comparisons.png

It's possible to compare any audio codec to another one, regardless of technology.

In your first comment, by comparable, did you mean objectively or subjectively?

Re: TSAC: Very Low Bitrate Audio Compression

Reply #32 –
@2012
I've indicated both examples of  subjective and objective studies from developers.
You just cropped and quoted my last message to only objective study for some particular reason.

Also It's trivial to provide ABC/HR or ABX logs for TSAC vs "any other format".  And Kamedo2 is already preparing the test  :)

Re: TSAC: Very Low Bitrate Audio Compression

Reply #34 –
Quote from: itisljar on
Well, download it and try for yourself. I've tried it with "Gimme hope Joanna" because reasons :)
Sorry, I don't have any discrete GPU in my system.

Maybe I'll try when an OpenVINO-optimized version will be released.

EDIT
For those who are interested in ML-based audio codecs, I've collected some open source resources/codes in AUDIO category \ AI-based page \ Codecs subsection. Enjoy.
Hybrid Multimedia Production Suite will be a platform-indipendent open source suite for advanced audio/video contents production.
Official git: https://forart.it/HyMPS/

Re: TSAC: Very Low Bitrate Audio Compression

Reply #36 –
Also, a question: Maybe "TSAC" is named fully as "Transformer Audio Codec" because it uses a transformer model to shrink even more while keeping the quality high?

I compressed a 16-second clip of a song: "Vluarr - They Don't Know" to TSAC @ 3 kbps (setting: -q 5, file size: 6.1 KB) using a Intel i5-4460 CPU (was a pain on compression speed because I don't have a Nvidia GPU, but it worked OK).

Full command line:

Code: [Select]
E:\tsac-2024-04-08-win64>tsac -m E:\tsac-2024-04-08-win64\dac_stereo_q8.bin -M E:\tsac-2024-04-08-win64\tsac_stereo_q8.bin -q 5 c "C:\Users\Public\New folder (14)\New folder (2)\New folder (2)\y2mate.com - Vluarr  They Dont Know_44k_s_flac_16sec.wav" "C:\Users\Public\New folder (14)\New folder (2)\New folder (2)\output_q5.tsac"
E:\tsac-2024-04-08-win64>tsac -m E:\tsac-2024-04-08-win64\dac_stereo_q8.bin -M E:\tsac-2024-04-08-win64\tsac_stereo_q8.bin d "C:\Users\Public\New folder (14)\New folder (2)\New folder (2)\output_q5.tsac" "C:\Users\Public\New folder (14)\New folder (2)\New folder (2)\output_q5.wav"

Please check input and output (compressed as FLAC, because it fits within the 30 MB of files per post limit for the forum). ABX them, and listen them carefully. Post the results when done!

Also attached: TSAC version of the output.

Re: TSAC: Very Low Bitrate Audio Compression

Reply #37 –
Quote from: Kamedo2 on
I am conducting the listening test of the TSAC, and comparison is possible.

Cool.

It's probably tool late, but I'd be interested in how the attached sample would fare compared to others.
I deleted my TSAC encodes, but I remember the whispers being especially affected/overshadowed.
And a part of the instrumental play got too messy at lower qualities.

Re: TSAC: Very Low Bitrate Audio Compression

Reply #38 –
Quote from: MihaiPopa12346 on
Please check input and output (compressed as FLAC, because it fits within the 30 MB of files per post limit for the forum). ABX them, and listen them carefully. Post the results when done!
Code: [Select]
foo_abx 2.2.1 report
foobar2000 v2.2 preview 2024-05-01
2024-05-06 17:12:22

File A: input.flac
SHA1: bd824f975ffb8eaa36eedef18a8fba79dc40eb39
File B: output_q5.flac
SHA1: 5b197642f52a03d74215a21597c93cc826a93b37

Output:
XAudio2 : Default Sound Device
Crossfading: NO

17:12:22 : Test started.
17:15:03 : Test restarted.
17:15:03 : 01/01
17:15:08 : Test restarted.
17:15:08 : 02/02
17:15:15 : Test restarted.
17:15:15 : 03/03
17:15:22 : Test restarted.
17:15:22 : 04/04
17:15:28 : Test restarted.
17:15:28 : 05/05
17:15:33 : Test restarted.
17:15:33 : 06/06
17:15:38 : Test restarted.
17:15:38 : 07/07
17:15:43 : Test restarted.
17:15:43 : 08/08
17:15:43 : Test finished.

 ---------- 
Total: 8/8
p-value: 0.0039 (0.39%)

 -- signature -- 
22abf4db7b1e2113f428dde825de5fe41b6c6325
During the first second the input sounds more metallic. The codec simplified the output. But I don't think this kind of thing is even meant to be transparent. Just a tech demo to squish things slowly into tiny space, modern day VQF.

Re: TSAC: Very Low Bitrate Audio Compression

Reply #39 –
Also I attached a speech sample compressed as TSAC @ 3 kbps (setting: -q 4, file size: 1.3 KB) using a Intel i5-4460 CPU (was a pain on compression speed because I don't have a Nvidia GPU, but it worked OK).

Full command line:

Code: [Select]
E:\tsac-2024-04-08-win64>tsac -m E:\tsac-2024-04-08-win64\dac_mono_q8.bin -M E:\tsac-2024-04-08-win64\tsac_mono_q8.bin -q 4 c "C:\Users\Public\New folder (14)\New folder (2)\New folder (2)\New folder (3)\New folder (2)\speech_female_input.wav" "C:\Users\Public\New folder (14)\New folder (2)\New folder (2)\New folder (3)\New folder (2)\speech_female_input.tsac"
E:\tsac-2024-04-08-win64>tsac -m E:\tsac-2024-04-08-win64\dac_mono_q8.bin -M E:\tsac-2024-04-08-win64\tsac_mono_q8.bin d "C:\Users\Public\New folder (14)\New folder (2)\New folder (2)\New folder (3)\New folder (2)\speech_female_input.tsac" "C:\Users\Public\New folder (14)\New folder (2)\New folder (2)\New folder (3)\New folder (2)\speech_female_output.wav"

Made with Elevenlabs (Glinda voice).

What I feed in TTS: "Is anyone compressing music and speech in such tiny space?"

Please check input and output (compressed as FLAC, because it fits within the 30 MB of files per post limit for the forum). ABX them, and listen them carefully. Post the results when done!

Also attached: TSAC version of the output.

Re: TSAC: Very Low Bitrate Audio Compression

Reply #41 –
@fab7 ...what about a Gcolab-notebook to perform compression tests ?

EDIT:
What about a NN-based audio codecs blind comparison ?
I've collected some here.
Hybrid Multimedia Production Suite will be a platform-indipendent open source suite for advanced audio/video contents production.
Official git: https://forart.it/HyMPS/

Re: TSAC: Very Low Bitrate Audio Compression

Reply #43 –
Quote from: forart.eu on
@fab7 ...what about a Gcolab-notebook to perform compression tests ?

EDIT:
What about a NN-based audio codecs blind comparison ?
I've collected some here.

Just wanted to say thank you for compiling this.

I'm really excited about the potential of NN based audio encoders. I think concerns about power consumption will be less of a problem as upcoming CPUs and SoCs will have neural acceleration built in (if they haven't already).

My guess is that over time there will be a number of standard weights (hopefully static and reproducible) that are bundled with various encoders, perhaps as part of the operating system itself or in the formats specification.
rc55.com - nothing going on

Re: TSAC: Very Low Bitrate Audio Compression

Reply #44 –
I had an idea, but I'm unsure if it's a viable solution to improve things further. What if we use a lossy codec with a proven track record to compress audio files, then compare the result with the original and extract the differences? We could then encode these differences using a codec like this to achieve a sort of lossless quality while reducing the required resource requirements and compression time associated with computationally heavy codecs like this one. I'm curious to know if this hybrid approach could lead to a versatile solution that offers the best of both worlds.
 

Re: TSAC: Very Low Bitrate Audio Compression

Reply #45 –
Quote from: neoOpus on
I had an idea, but I'm unsure if it's a viable solution to improve things further. What if we use a lossy codec with a proven track record to compress audio files, then compare the result with the original and extract the differences? We could then encode these differences using a codec like this to achieve a sort of lossless quality while reducing the required resource requirements and compression time associated with computationally heavy codecs like this one. I'm curious to know if this hybrid approach could lead to a versatile solution that offers the best of both worlds.

I don't think this would really benefit as ultimately you are creating a lossy file so the differencing / difference encoding pass doesn't make sense. I suppose stacking different technologies is a curious one, reminds me of SBR or switching between CELT/SILK in Opus depending on content.

Perhaps blending codecs would be an interesting idea - you could have 0-16khz covered by Opus or similar and anything above that covered by a trained NN model for less sensitive frequency ranges, much like how TSAC works. I'm not a domain expert on that though.
rc55.com - nothing going on