HydrogenAudio

Hydrogenaudio Forum => Scientific Discussion => Topic started by: enzo on 2018-07-21 12:12:54

Title: SuperFast LAME multi-threaded MP3 encoder
Post by: enzo on 2018-07-21 12:12:54
Hi folks,

I just published the 3rd preview release of fre:ac with my SuperFast multi-threading technology. You can download it from GitHub (https://github.com/enzo1982/superfast#superfast-codecs).

This release adds support for multi-threaded LAME MP3 encoding. It scales very well with the number of CPU cores and can achieve a 3.5x speed-up on a quad core CPU. On my i7-6900K, I was able to measure up to 12x speed-up when running with all 16 threads (in most scenarios, however, either decoding or HDD/SSD speed will be a bottleneck before going to such speeds).

Two key points distinguish SuperFast LAME from previous attempts to build multi-threaded MP3 encoders:


Making it work while keeping the bit-reservoir enabled was actually the greatest challenge. Here is a technical article on how this is implemented: SuperFast LAME technical details (https://freac.org/developer-blog-mainmenu-9/14-freac/287-superfastlame)

I plan to implement this technology on top of the command line LAME frontend in the future. For now, my priority is on releasing fre:ac 1.1 beta and final versions, though.

Thanks in advance for trying the preview and posting your results and opinions!
Title: Re: SuperFast LAME multi-threaded MP3 encoder
Post by: punkrockdude on 2018-07-23 09:36:16
Interesting!
Title: Re: SuperFast LAME multi-threaded MP3 encoder
Post by: Compact Dick on 2018-07-24 11:24:46
Bravo! I shall try this out in the upcoming weeks and let you know of the results.
Title: Re: SuperFast LAME multi-threaded MP3 encoder
Post by: Kraeved on 2024-04-19 20:34:46
This topic didn't get much attention, and the Github repository was archived on December 11, 2020. Why, @enzo?

As far as I understand, MP3 format, due to its design (the notorious reservoir of bits), does not favor multicore encoding, but you, describing the technical details (https://freac.org/developer-blog-mainmenu-9/14-freac/287-superfastlame), stated that it was possible without loss of quality. I encoded two WAV PCM files (44.1 kHz 16 bit stereo and 11025 Hz 16 bit mono) using SuperLame as follows: freaccmd.exe in.wav -e superlame.

The latter one failed.
Title: Re: SuperFast LAME multi-threaded MP3 encoder
Post by: enzo on 2024-04-20 15:37:54
This topic didn't get much attention, and the Github repository was archived on December 11, 2020. Why, @enzo?
The SuperFast code was integrated into official fre:ac/BoCA releases and development continued there, so there was no need for a separate repository anymore. The SuperFast LAME code is here (https://github.com/enzo1982/BoCA/tree/master/components/encoder/lame) now.

As far as I understand, MP3 format, due to its design (the notorious reservoir of bits), does not favor multicore encoding, but you, describing the technical details (https://freac.org/developer-blog-mainmenu-9/14-freac/287-superfastlame), stated that it was possible without loss of quality. I encoded two WAV PCM files (44.1 kHz 16 bit stereo and 11025 Hz 16 bit mono) using SuperLame as follows: freaccmd.exe in.wav -e superlame.

The latter one failed.
There were some bugs fixed in the BoCA repo after the original repo had been archived. Current fre:ac seems to encode this file correctly.
Title: Re: SuperFast LAME multi-threaded MP3 encoder
Post by: Kraeved on 2024-04-20 16:46:45
There were some bugs fixed in the BoCA repo after the original repo had been archived. Current fre:ac seems to encode this file correctly.

I tried freac-1.1.7-windows-x64.zip (https://www.freac.org/downloads-mainmenu-33) (2023-05-05) and the previous command did not work. Oh, you removed SuperLame from the list of encoders in favor of a dedicated flag --superfast. I applied it and witnessed some obscure warning. It would be nice to get a more detailed report on the encoding settings used.

Code: [Select]
$ freaccmd.exe history.wav -e superlame
Encoder 'superlame' is not supported by fre:ac!

$ freaccmd.exe
...
Encoder <id> can be one of:
        ffmpeg-alac, fdkaac, flac, lame, meh, mac, mpc, vorbis,
        ofr, opus, speex, tak, wv, wma, sndfile-wave, sndfile

$ freaccmd.exe history.wav -e lame --superfast
Warning: APIC ID are not supported, core count can be wrong if SMT is disabled and cache instances count will not be available.
Processing file: history.wav...done.