Skip to main content

Topic: Lossless Extensions for Opus (Backwards Compatible) (Read 43950 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
Lossless Extensions for Opus (Backwards Compatible)
After reading a good chunk of the Opus RFC, I believe it may be possible to use Opus padding to embed lossless deltas for each packet.  It states that this padding may be of any size, and that while an Opus encoder itself must set any padding to zero, the decoder must accept any value.  It also states that when the decoder has finished reading bytes from a frame for decoding, it may not spill over into the padding for further input.  Hrm...  Sounds like an arbitrary extension field to me.

My only concern is about just how much padding decoders would be willing to accept before deciding that the packet is a DOS attack.

  • NullC
  • [*][*][*]
  • Developer
Lossless Extensions for Opus (Backwards Compatible)
Reply #1
to embed lossless deltas for each packet.
Deltas are not going to do you any good. The opus specification is quite intentionally not bit exact, and the decoder gives slightly different output on different systems and different versions of the decoder. The design of the format also does not reliably result in compact simple lossless deltas (e.g. the phase shift in hybrid mode, or how HF gets filled with 'folded' noise).

For completeness, the RFC also specifies that the padding MUST be set to all zeros— though certainly some future revision to the RFC could make use of the freedom that comes from also mandating that the decoder ignore the padding.
  • Last Edit: 24 February, 2013, 03:00:40 PM by NullC

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Lossless Extensions for Opus (Backwards Compatible)
Reply #2
The opus specification is quite intentionally not bit exact, and the decoder gives slightly different output on different systems and different versions of the decoder.


This is easily solvable: mandate that decoders that want to use this lossless extension use a specified decoder that *is* bit exact with some reference (for example a fixed point implementation) before applying the correction data.

Old decoders will decode not-bit exact and sound fine, and the new ones will decode bit exact and know how to apply the correction info.
  • Last Edit: 24 February, 2013, 03:11:00 PM by Garf

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Lossless Extensions for Opus (Backwards Compatible)
Reply #3
And how to losslessly encode 44.1 kHz audio?

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Lossless Extensions for Opus (Backwards Compatible)
Reply #4
And how to losslessly encode 44.1 kHz audio?


Ah, good point. NullC also pointed out to me on IRC this setup will fail while seeking.

Lossless Extensions for Opus (Backwards Compatible)
Reply #5
When I came up with this idea, my primary use case was movies.  With that said, I can't say that I really care about encoding 44.1 kHz audio.  Is this idea worthless without this feature?

Also, I'm wondering if DTS doesn't already hold a patent on extendable audio streams (DTS, DTS ES, DTS 96/24).
  • Last Edit: 24 February, 2013, 04:08:37 PM by wswartzendruber

  • saratoga
  • [*][*][*][*][*]
Lossless Extensions for Opus (Backwards Compatible)
Reply #6
When I came up with this idea, my primary use case was movies.  With that said, I can't say that I really care about encoding 44.1 kHz audio.  Is my idea worthless without this feature?


Why Opus?  Something like Wavpack would probably work more efficiently unless you want ultra-low bitrate lossy files.

Lossless Extensions for Opus (Backwards Compatible)
Reply #7
Compatibility with existing Opus decoders.

  • db1989
  • [*][*][*][*][*]
  • Global Moderator
Lossless Extensions for Opus (Backwards Compatible)
Reply #8
Which, as has already been explained, is not possible.

Opus was not designed as a lossless format. If they had wanted it to have that capability in the future, they would not have implemented various features that rule out such usage. In other words, either it was never considered as an option, or it was considered and dismissed on technical or other grounds.
  • Last Edit: 24 February, 2013, 04:15:06 PM by db1989

Lossless Extensions for Opus (Backwards Compatible)
Reply #9
Nowhere in this thread has it been explained that this isn't possible.  NullC has said that RFC6716 encoders are to set any padding to zero.  But this isn't going to be an RFC6716 encoder.  I aim to write a custom encoder that generates lossless streams that can still be played back by RFC6716 decoders.

Do DTS-ES encoders follow the specs for DTS encoders?  Heck no.  But they're still compatible.

EDIT:  Now with that said, some good points have been raised and there are issues that need to be solved.  Bit-exact inaccuracy is the main one.
  • Last Edit: 24 February, 2013, 04:32:25 PM by wswartzendruber

  • saratoga
  • [*][*][*][*][*]
Lossless Extensions for Opus (Backwards Compatible)
Reply #10
Compatibility with existing Opus decoders.


Yeah, but why?

Lossless Extensions for Opus (Backwards Compatible)
Reply #11
Compatibility with existing Opus decoders.


Yeah, but why?

To have a single encode that you can use on the greatest number of devices.  Looking at the parties backing Opus, I see it becoming a widely-supported codec.  Consider Dolby Digital EX and DTS-ES.  They would have never taken off had they not been compatible with existing implementations of their respective decoders.

In order to convince me this isn't a good idea, I would need to be pointed to a widely-supported lossless audio codec, or one that will arguably become this way in the near future.

  • saratoga
  • [*][*][*][*][*]
Lossless Extensions for Opus (Backwards Compatible)
Reply #12
Mp3, aac.

Lossless Extensions for Opus (Backwards Compatible)
Reply #13
Mp3, aac.

Why yes, MP3 does have lossless extensions.  It's also heavily patented.

  • saratoga
  • [*][*][*][*][*]
Lossless Extensions for Opus (Backwards Compatible)
Reply #14
Do you really care?  The MP3 patent license is almost always paid anyway on hardware devices (e.g. android, dvd players, etc) and no one is going to go after some niche hybrid lossless format for royalties since theres no money in it.

It sounded like you wanted compatibility.  If so, just use MP3 or AAC.
  • Last Edit: 25 February, 2013, 06:55:00 AM by db1989

Lossless Extensions for Opus (Backwards Compatible)
Reply #15
mp3HD can't go beyond stereo and 16-bit.  I'm also unaware of any lossless extensions for AAC.  And I do care about patents because this isn't just about me.  I would also like for others to be able to use this, as well.
  • Last Edit: 25 February, 2013, 06:55:06 AM by db1989

  • saratoga
  • [*][*][*][*][*]
Lossless Extensions for Opus (Backwards Compatible)
Reply #16
mp3HD can't go beyond stereo and 16-bit.  I'm also unaware of any lossless extensions for AAC.


There are actually lossless extensions for both, but aren't you planning to implement your own system?  Who cares what already exists?

And I do care about patents because this isn't just about me.  I would also like for others to be able to use this, as well.


Realistically the number of people who would ever use something like this is so small you will never attract enough attention for patent licensing to matter.

Lossless Extensions for Opus (Backwards Compatible)
Reply #17
There are actually lossless extensions for both, but aren't you planning to implement your own system?  Who cares what already exists?

Realistically the number of people who would ever use something like this is so small you will never attract enough attention for patent licensing to matter.

Interestingly enough, you answered your own question.  The new system must remain compatible with existing decoders precisely because nobody will have a new one.  At least at first.

Think of it like this, "Hey, here's this lossless codec you can use, and it still plays on anything that can decode Opus."
  • Last Edit: 24 February, 2013, 07:44:49 PM by wswartzendruber

  • saratoga
  • [*][*][*][*][*]
Lossless Extensions for Opus (Backwards Compatible)
Reply #18
There are no existing hybrid lossless decoders for Opus, so if your goal is to be compatible with an existing lossless implementation (which is not what you said above but what you seem to be implying here), you cannot use Opus.  You would have to use the existing Mp3 or AAC variants.

If your goal is to be compatible only with an existing lossy format, then you can use any of the above mentioned formats, either extending it to your needs or simply using existing software and standards.  In this case, I'd probably recommend just using MPEG-4 SLS, as it is backwards compatible with anything that supports AAC and is already standardized.  You could also use MP3 and simply define your own correction format, but since you want 5.1 support, it would probably be easier to use AAC since there is already widespread support for 5.1 AAC.
  • Last Edit: 25 February, 2013, 05:14:05 AM by db1989

Lossless Extensions for Opus (Backwards Compatible)
Reply #19
There are no existing hybrid lossless decoders for Opus, so if your goal is to be compatible with an existing lossless implementation...


Now I think I see the problem.  There is a fundamental misunderstanding of what I am saying.  Here is the principal of operation for the encoder:

1. Take the current chunk of input PCM and use an Opus encoder to generate a compressed, lossy, Opus packet.
2. Decode this packet (probably with the fixed-point decoder) and hang on to the PCM values it returns.
3. Subtract these PCM values from the input PCM values and compress those with a lossless codec.
4. Store this this new, losslessly-compressed chunk of data in the Opus padding area.

Here's the principal of operation for a standard Opus decoder:

1. Take the current input Opus packet and decode it to PCM.
2. Ignore the padding.

Here's the principal of operation for the decoder I would make.

1. Take the current input Opus packet and decode it to PCM with the fixed-point decoder.
2. Take the padding and feed it into the lossless decoder.
3. Add the decoded Opus value and the decoded lossless value together and store it as output PCM.

Does this make more sense now?
  • Last Edit: 24 February, 2013, 09:11:14 PM by wswartzendruber

Lossless Extensions for Opus (Backwards Compatible)
Reply #20
I also missed the part above where this proposition will mess up seeking.  I'm hoping someone can explain why.  That seems more like a container issue to me.

  • jmvalin
  • [*][*][*][*][*]
  • Developer
Lossless Extensions for Opus (Backwards Compatible)
Reply #21
I also missed the part above where this proposition will mess up seeking.  I'm hoping someone can explain why.  That seems more like a container issue to me.


It's not a container issue. if you seek within an Opus file, what you decode from that point will not be bit-exact with a decoder that started from the beginning of the file. The difference is so small that it's not audible, but it still means losing the lossless property as soon as you seek.

But really, I think most comments have so far missed the real issue here: A lossless extension to Opus would be mostly pointless. A 48 kHz 16-bit PCM stereo file is 1536 kb/s, and on average a codec like FLAC will compress about 50%, so that means 768 kb/s. If you consider that Opus gives you pretty good quality at 96 kb/s, then it means that Opus+FLAC would cost you 864 kb/s. OTOH, a lossless extension to Opus, would compress slightly worse then than a "native" lossless codec like FLAC (partly due to things like intensity stereo and spectral folding), probably costing us about at *least* 5%, meaning that we'd end up at 806 kb/s. So in the end, we have a messy new extension that saves us only 7% compared to just carrying two sets of files. There's just no point, really. Get over it.
  • Last Edit: 24 February, 2013, 11:43:52 PM by jmvalin

Lossless Extensions for Opus (Backwards Compatible)
Reply #22
Your first paragraph raises a very good and valid point.  This is a technical issue that I will have to assess.

Your second paragraph, on the other hand, is mainly a matter of philosophy.  You also appear to be a bit confused.  "We" aren't going to be having any extension.  "I" will be having an extension that others will be free to use.  I do not require your blessing to continue, nor quite frankly, do I care if I have it.  Especially not with that attitude.

I am looking for technical issues.  So far, two very good ones have been raised.
  • Last Edit: 25 February, 2013, 05:14:51 AM by db1989

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
Lossless Extensions for Opus (Backwards Compatible)
Reply #23
we have a messy new extension that saves us only 7% compared to just carrying two sets of files.
..and sometimes it's worse, and it's always worse than just a pure lossless file that doesn't depend on a lossy core.

If you must go down that route and want to hack around with something "free", WavPack Hybrid and lossyWAV/lossyFLAC + correction files are built with this in mind.

Cheers,
David.

Lossless Extensions for Opus (Backwards Compatible)
Reply #24
Why does DTS-HD have a lossy core stream?