Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Google Lyra goes to open source (C++) (Read 7137 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Google Lyra goes to open source (C++)

Hi to all.

- Lyra - enabling voice calls for the next billion users
- GitHub

------------------
The Open Source Release
The Lyra code is written in C++ for speed, efficiency, and interoperability, using the Bazel build framework with Abseil and the GoogleTest framework for thorough unit testing. The core API provides an interface for encoding and decoding at the file and packet levels. The complete signal processing toolchain is also provided, which includes various filters and transforms. Our example app integrates with the Android NDK to show how to integrate the native Lyra code into a Java-based android app. We also provide the weights and vector quantizers that are necessary to run Lyra.

We are releasing Lyra as a beta version today because we wanted to enable developers and get feedback as soon as possible. As a result, we expect the API and bitstream to change as it is developed. All of the code for running Lyra is open sourced under the Apache license, except for a math kernel, for which a shared library is provided until we can implement a fully open solution over more platforms. We look forward to seeing what people do with Lyra now that it is open sourced. Check out the code and demo on GitHub, let us know what you think, and how you plan to use it!
------------------

 

Re: Google Lyra goes to open source (C++)

Reply #1
It would need a file container to be usable outside of app-app

Re: Google Lyra goes to open source (C++)

Reply #2
Lyra 0.0.2 was released on June 28th.

Quote
Lyra version 0.0.2 is now available on GitHub. The main improvement of this version is the open-source release of the sparse_matmul library code, which was co-developed by Google and DeepMind. That means no more pre-compiled “.so” dynamic library binaries and no more restrictions on which toolchain to use, which opens up the door to port Lyra onto different platforms. The full list of features and fixes include:

  • Release sparse_matmul library code and remove pre-compiled dynamic library binaries.
  • Add support for the Bazel default gcc toolchain on linux, and make this the default instead of the clang toolchain.
  • Fix noise bursts at the beginning of output audio files.
  • Abstract out UnitFloatToInt16Scalar, UnitFloatToInt16 and Int16ToUnitFloat functions.
  • Provide operator<< to unique_ptr to be used with CHECK() macros.
  • Fix float distribution compatibility in benchmark_decode_lib.

However, according to an issue on the bug tracker, "the samples in the AI blog were produced by our latest Lyra model, which is one generation ahead, but will be open-sourced as soon as some technical details are ironed out."

And a later comment added that "Lyra v0.0.2 did not ship any quality improvements except for fixing the noise bursts at the beginning of the audio samples (#46)."

Re: Google Lyra goes to open source (C++)

Reply #3
according to an issue on the bug tracker, "the samples in the AI blog were produced by our latest Lyra model, which is one generation ahead, but will be open-sourced as soon as some technical details are ironed out."
The choice of earlier or later neural network (NN) model might make a big difference. I just stumbled across this peer-reviewed paper comparing NN enhanced speech codecs with more traditional speech codecs, including Lyra (the ealier version, I assume), at very low bit-rates. The corresponding figure from the paper is attached.

The audio quality achieved by the best performing NN based codec, "SSMGAN", is impressive (I've never seen anything close to these MUSHRA scores at only 1.6 kbps).

The performance of Lyra is not impressive currently, I'm afraid. Will be interesting to see/hear how much better it can be once their updated NN model is publicly available).

Chris
If I don't reply to your reply, it means I agree with you.



Re: Google Lyra goes to open source (C++)

Reply #6
I'm a little disappointed the music samples were so short, but excited to see where things go next.
rc55.com - nothing going on

Re: Google Lyra goes to open source (C++)

Reply #7
The music samples in Lyra are fun, the way they make a person sing or say ghostly words. "Oh now..." I'd like to listen for longer fragments just for entertainment.

Re: Google Lyra goes to open source (C++)

Reply #8
This is amazing, I can't wait to hear this working in mainstream VOIP apps.

Re: Google Lyra goes to open source (C++)

Reply #9
Yeah, cool, we can VOIP over dialup again! But it uses a huge amount of processing power to work. Or so I'd assume. Maybe I'm wrong about that.

Re: Google Lyra goes to open source (C++)

Reply #10
https://arxiv.org/pdf/2107.03312.pdf

SoundsStream has low-complexity mode which requires ~5% of smartphone single core (and it's already now at very early stage).

So, this LC mode@ 6kbps sounds as good as normal mode at 4kbps. And this LC@6 kbps is still comparable to EVS 9.6-13.2 kbps and close Opus 16 kbps. 

P.S. Microsoft has already implemented their neural codec Satin which is used in MS Teams.  So it's doable and complexity is affordable already now.
I think MSFT was first here or at least one of the very first to integrate neural codecs into a real products.

This is amazing, I can't wait to hear this working in mainstream VOIP apps.
Not only VOIP apps as SoundStream is general audio codec. According to the paper music quality scales very nicely with bitrate increase.

Re: Google Lyra goes to open source (C++)

Reply #11
Nice, but i am extremely surprised i am the first one to ask where is the encoder

Re: Google Lyra goes to open source (C++)

Reply #12
Lyra V2 - a better, faster, and more versatile speech codec
Friday, September 30, 2022

Quote
New Architecture

Lyra V2 is based on an end-to-end neural audio codec called SoundStream. [...] In Lyra V2, we support three different bitrates: 3.2 kps, 6 kbps, and 9.2 kbps. [...]

Better Performance

With the new architecture, the delay is reduced from 100 ms with the previous version to 20 ms. In this regard, Lyra V2 is comparable to the most widely used audio codec Opus for WebRTC, which has a typical delay of 26.5 ms, 46.5 ms, and 66.5 ms.

Lyra V2 also encodes and decodes five times faster than the previous version. On a Pixel 6 Pro phone, Lyra V2 takes 0.57 ms to encode and decode a 20 ms audio frame, which is 35 times faster than real time. The reduced complexity means that more phones can run Lyra V2 in real time than V1, and that the overall battery consumption is lowered.

Higher Quality

Our listening tests show that the audio quality (measured in MUSHRA score, an indication of subjective quality) of Lyra V2 at 3.2 kbps, 6 kbps, and 9.2 kbps measures up to Opus at 10 kbps, 13 kbps, and 14 kbps respectively. [...] While Lyra V1 already compares favorably to the Adaptive Multi-Rate (AMR-NB) codec, Lyra V2 further outperforms Enhanced Voice Services (EVS) and Adaptive Multi-Rate Wideband (AMR-WB), and is on par with Opus, all the while using only 50% - 60% of their bandwidth.

Re: Google Lyra goes to open source (C++)

Reply #13
Lyra V2 - a better, faster, and more versatile speech codec
Friday, September 30, 2022

Quote
New Architecture

Lyra V2 is based on an end-to-end neural audio codec called SoundStream. [...] In Lyra V2, we support three different bitrates: 3.2 kps, 6 kbps, and 9.2 kbps. [...]

Better Performance

With the new architecture, the delay is reduced from 100 ms with the previous version to 20 ms. In this regard, Lyra V2 is comparable to the most widely used audio codec Opus for WebRTC, which has a typical delay of 26.5 ms, 46.5 ms, and 66.5 ms.

Lyra V2 also encodes and decodes five times faster than the previous version. On a Pixel 6 Pro phone, Lyra V2 takes 0.57 ms to encode and decode a 20 ms audio frame, which is 35 times faster than real time. The reduced complexity means that more phones can run Lyra V2 in real time than V1, and that the overall battery consumption is lowered.

Higher Quality

Our listening tests show that the audio quality (measured in MUSHRA score, an indication of subjective quality) of Lyra V2 at 3.2 kbps, 6 kbps, and 9.2 kbps measures up to Opus at 10 kbps, 13 kbps, and 14 kbps respectively. [...] While Lyra V1 already compares favorably to the Adaptive Multi-Rate (AMR-NB) codec, Lyra V2 further outperforms Enhanced Voice Services (EVS) and Adaptive Multi-Rate Wideband (AMR-WB), and is on par with Opus, all the while using only 50% - 60% of their bandwidth.

Great news!  where is the encoder?

Re: Google Lyra goes to open source (C++)

Reply #14
You need to go to https://github.com/google/lyra and follow the instructions. It's an experimental codec, so you need to compile the executables (if there are any, maybe it's just a library at the moment?) yourself.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Google Lyra goes to open source (C++)

Reply #15
Could someone be so kind to compile Lyra as .exe for Windows 7+ and attach it here, please?
Not everyone is a C developer with proper skills and tools.
• We can do better about lossy music: Opus complexity & qAAC dependence on Apple is a departure from Vorbis & Musepack breakthroughs
• Clipping-free MP3 encoding for vintage gear: SoX to 44.1 kHz → LoudMax -1 dB ISP → ADClip8 → Smart dither 16 bit → Lame3995o -Q1
• Plz, update WavPack hybrid & FSLAC

Re: Google Lyra goes to open source (C++)

Reply #16
On the other hand, what's the use of a compiled Lyra encoder if Foobar2000 can't play that file?
If only we decompress it back to WAV and compress it into something regular like Vorbis.
• We can do better about lossy music: Opus complexity & qAAC dependence on Apple is a departure from Vorbis & Musepack breakthroughs
• Clipping-free MP3 encoding for vintage gear: SoX to 44.1 kHz → LoudMax -1 dB ISP → ADClip8 → Smart dither 16 bit → Lame3995o -Q1
• Plz, update WavPack hybrid & FSLAC

Re: Google Lyra goes to open source (C++)

Reply #17
In case you are still interested, this is a build I made last May when I was experimenting with it.  I don't know if it works on Windows 7 though.  Maybe you can try it and let us know.

Re: Google Lyra goes to open source (C++)

Reply #18
@autodidact Thank you, it works on my end. However, I had to create Lyra folder and move Model_coeffs from your archive down there, otherwise it threw an error. The encoder is finicky about the input sample rate (22.05 and 44.1 kHz are not accepted, so I used 32 kHz). And the input must be mono. I managed to compress and decompress using only the default bitrate 3200, whereas some sound garbage came out on 6000 and 9200.

Original samples:
* Blue Monday FM - Bee moved (music with vocals, 0:40, 96 kHz 24 bit FLAC)
* Nick Dolle - Doctor quotes (speech, 0:23, 44.1 kHz 16 bit WAV)

Files were pre-processed as needed via Foobar2000 DSP (mono, 32 kHz, 16 bit) and then sent into Lyra.

Size difference (bytes):
* 2 555 108 → 15 944
* 1 468 922 → 9 176

I consider the bitrate 3200 suitable for communication between polar explorers and cosmonauts, as well as for keeping an audio diary in such harsh conditions when the communication channel and storage space are quite limited, but for a long listening session of an audio book (Paul Bowles - The sheltering sky, narrated by Jennifer Connelly for ~10 hours) it is more appropriate to use another codec. Perhaps I'll change my mind if I get a chance to test Lyra's higher bitrates.

• We can do better about lossy music: Opus complexity & qAAC dependence on Apple is a departure from Vorbis & Musepack breakthroughs
• Clipping-free MP3 encoding for vintage gear: SoX to 44.1 kHz → LoudMax -1 dB ISP → ADClip8 → Smart dither 16 bit → Lame3995o -Q1
• Plz, update WavPack hybrid & FSLAC