About
fat_llama is a Python package for upscaling audio files to FLAC or WAV formats using advanced audio processing techniques. It utilizes CUDA-accelerated calculations to enhance audio quality by upsampling and adding missing frequencies through FFT (Fast Fourier Transform), resulting in richer and more detailed audio.

Features

• Upscale MP3 files to high-quality FLAC format.
• Optional iterative soft thresholding (IST) for enhanced audio processing.
• Gain adjustment, equalization, and optional Wiener filtering.
• Supports GPU-accelerated processing with CuPy.

How it works


Algorithm Explanation
The upscaling process involves several steps:

• Reading Audio File: The audio file is read, and the audio samples are extracted along with the sample rate and bitrate.
• Calculating Upscale Factor: The upscale factor is calculated to achieve the target bitrate.
• Upscaling Channels: The audio channels are upscaled using an interpolation algorithm. Each sample is repeated multiple times to increase the resolution.
• Iterative Soft Thresholding (IST): IST is applied to enhance the audio by adding missing frequencies. This process uses FFT to transform the signal into the frequency domain, apply a threshold to keep significant frequencies, and then inverse transform back to the time domain.
• Scaling Amplitude: The amplitude of the upscaled audio is scaled to match the original.
• Normalizing Audio: The audio is normalized to the range -1 to 1.
• Writing FLAC File: The processed audio is written to a FLAC file.

Spectrogram Results

• During a lossy audio treating, the best approach is to carefully decode and preserve it as much as possible: that's why a floating point internal computing and output grants better fidelity;
• Operations like normalization (that alters the level of the original signal) should be optional and not part of the process;
• Since your software relies on FFMPEG, I strongly suggest you to always use the -drc_scale 0 parameter when decoding lossy sources (even if it's AC3-specific feature);
• Always seek for new scientific papers/code - such as @zawi01's Audio Dequantization Using (Co)Sparse (Non)Convex Methods - that may help to achieve better results;
• While you've probably already done this, I recommend you to check how Izotope' Spectral Recovery and Stereotool's Delossifier functions works, for possible inspirations.

...anyway nowdays the best approach is to "reconstruct" signal using machine learning IMHO.

Only the same as creating hi-res images from lo-res ones.  The reconstructed image may be crisp, but the detail is invented.  Won't the same be true of audio?  Do you want detail in the audio which was never there in the first place??

Quote from: fooball on 2024-07-30 23:43:45

Only the same as creating hi-res images from lo-res ones.  The reconstructed image may be crisp, but the detail is invented.  Won't the same be true of audio?  Do you want detail in the audio which was never there in the first place??

Pretty much this right here.  If it was lossy compressed to begin with then it can't be made lossless because those details are gone.  Fat Llama is just another snake's oil scam to me.  Doesn't matter if it uses a traditional algorithm or an AI based algorithm, still the same, tired, old thing.

Given that regenerating the original signal "simplified" by a lossy encoding is obviously impossible, GANs works quite differently from everything else: check out the very interesting Stochastic Restoration of Heavily Compressed Musical Audio using Generative Adversarial Networks scientific paper to understand why.

Just a shot:

Are you saying the results sound pleasing?  I suppose there might be some value in that.

I assume these spectograms are from a different software than the one in the initial post.

Said that, while this software might make the sound more pleasant compared to the 32kbps version, I see that it is also partially destroying the harmonics. In all the cases, I see that tonal frequencies are less clear than in both, the original and the 32kbps cases, so partially buried in noise.

Of course the regenerated high end can also sound strange and not follow the sound (like, on cymbal sound different than the next cymbal)

[1.1.0] - 2024-08-01

Chanaged

• Moved adaptive filtering to after normalization and auto-scaling steps.
• Reduced step size for LMS adaptive filter for improved stability.
• Ensured all processing uses CuPy for GPU acceleration.
• Added detailed comments and logging for better traceability.

I would install the latest version of fat_lama=1.0.2.3 for this. You can run the conversion for mp3 to flac using converter.py. And the analysis.py to regenerate results. Cheers.

• Matthew McQuistion's upscalemp3 that "Converts an mp3 (lossy) file back into its uncompressed wav counterpart based on a generative AI model built with tensorflow."
• Detlef Kroll's Audio Delossifier which claims to "Delossify compressed audio (mp3 and others) with Python and Tensorflow"

And remember TOS #8.   The focus here is what you can hear (usually in a blind ABX test), not what you can measure or see in a graph.   It's easier to make a pretty spectrum than it is to get good sound.     (I'm not saying you're not hearing an improvement or difference with this tool.)