Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Lossless Extensions for Opus (Backwards Compatible) (Read 81114 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Lossless Extensions for Opus (Backwards Compatible)

Reply #75
I'm not going to stop until I see for myself that this will not work well.

This is an exemplary attitude! Sometimes I wish we had a bit more of this in the scientific and engineering world, since it's often the only way to achieve progress.

Honestly, good luck with your project. And please, do let us know about your findings.

Chris
If I don't reply to your reply, it means I agree with you.

Lossless Extensions for Opus (Backwards Compatible)

Reply #76
I'll just use this thread as sort of a log.

Anyway, I've tested Monkey's Audio against Opus residuals at a variety of bitrates and it is proving to be competitive against OptimFROG.  Given that I have no idea what lossless codec may eventually be used to compress the residuals, I need to redo the principles of operation to accommodate one that uses variable frame sizes (that is, other than 60 ms, which I have been assuming up until now).  It is highly likely that there will be many Opus packets for each delta packet, and this ratio is likely to vary throughout a stream.

Lossless Extensions for Opus (Backwards Compatible)

Reply #77
With mention of an open lossy codec being a requirement, the other option is Ogg Vorbis, which supports specific sampling rates and isn't too far behind Opus in bitrate efficiency and is an open format that has fairly widespread support in the wild (main exception being Apple, despite Wikipedia's use of Vorbis). Then you have no worry about resampling consistently so that the hybrid remains lossless. You then just need to presumably place non-Vorbis packets in the Ogg stream that all Vorbis decoders (except your hybrid decoder) will ignore. Presumably it would also be easy to strip out only Vorbis packets from the Ogg stream to obtain a smaller lossy file.

The other option where sampling rate can be specified and still be expected to play is Xiph.org's CELT (forerunner to the CELT half of Opus you're thinking of), but plain CELT decoder support in the wild is very limited and will very likely remain so now Opus has superceded it, even though I believe Google's plugin for Chat/Hangouts uses CELT in its new Studio mode (click Learn More in this link) and will possibly implement Opus in due course, possibly piggybacking on WebRTC integration.

On the positive side for Opus, it's possible that being standardized by IETF with clearly specified open patent licensing, support will eventually surpass that of Vorbis.
Dynamic – the artist formerly known as DickD

Lossless Extensions for Opus (Backwards Compatible)

Reply #78
Dynamic's suggestion is very interesting. In addition to the points mentioned there are a few other things I wanted rehearse here:

- former lossless Vorbis [q 10] discontinued;
- new iteration of Vorbis, possibly ditched due to Ghost/OPUS and the possibility of broken compatibility;
- the AoTuV project low-bitrate settings (q-1, q-2) sounded better to me than Speex at comparable bitrate, IMO.
"Something bothering you, Mister Spock?"

Lossless Extensions for Opus (Backwards Compatible)

Reply #79
One of the main reasons for choosing Opus is because of the IETF standardization Dynamic mentioned.  Also, Opus lets me extend it at the sub-container level.  There should be no reason a lossless/hybrid stream can't be placed in any container that can handle standardized Opus.  My hybrid encoder API so far is completely container agnostic.

Lossless Extensions for Opus (Backwards Compatible)

Reply #80
Florin got back to me.  He's just in a bit of a personal dilemma and estimates reviewing my comments in one week.

Lossless Extensions for Opus (Backwards Compatible)

Reply #81
One week, right.  I think he's in a bit more of a jam than he admitted.

Whatever.  As soon as libopus-1.1 comes out, I'm gonna start gutting the tree until I have a fixed-point decoder and nothing else.

Lossless Extensions for Opus (Backwards Compatible)

Reply #82
One week, right.  I think he's in a bit more of a jam than he admitted.

Whatever.  As soon as libopus-1.1 comes out, I'm gonna start gutting the tree until I have a fixed-point decoder and nothing else.

All right, now that I've graduated from college (Bachelor in Computer Science) and have myself a job, I can get back to the task at hand.  After having thought about this for a while, I think the best answer may just be to implement my own Opus (CELT) decoder in fixed-point arithmetic.  This is going to really suck, but I need stability with a base I understand.  I also don't see myself needing the SILK parts of the reference implementation.  I should know more this weekend after I've gone over the appropriate parts of the RFC.

Lossless Extensions for Opus (Backwards Compatible)

Reply #83
Florin finally got back to me!  He's been all tied up with real-life stuff.  Moving and whatnot.  He still seems determined to help.  I won't share what he told me because I don't have his permission to do so, but I'll share what I said to him:

Quote
Hi Florin,

I'm on my tablet so I won't type a lot.  I have outlined some main points below.

1. Thank you for getting back to me.

2. I am in the United States (California).

3. I have determined that we should have our own implementations of Opus.

4. We only need the MDCT-based CELT layer.

5. I don't know how to do any of this, so I am learning from the ground up.  I have just finished, as practice, my first FFT implementation, and then used that to create (from scratch) an FFT-based convolver that uses overlap-and-add.  In the coming days I will move onto learn and implement the MDCT transform.

6. I hate Jean-Marc for trying to shoot this down so fast, but he has a point.  Extension packets need to be standardized first.

7. I've lost the work I did, but it wasn't applicable anyway.  I've since started using Git.

8. I still have every intention of making this happen.

9. You can track my learning progress here:  https://github.com/wswartzendruber/logic/

-William

Lossless Extensions for Opus (Backwards Compatible)

Reply #84
This stuff died quickly.

Nevertheless i wonder why CELT was not designed in mind to be extensible to lossless. Most new video codecs support lossless coding as well (H.264, H.265, Daala) which is a much more complex area and still achievable. I wonder why audio codecs are not developed with this criteria in mind. Would it reduce CELT's efficiency considerably?

Lossless Extensions for Opus (Backwards Compatible)

Reply #85
This stuff died quickly.

Nevertheless i wonder why CELT was not designed in mind to be extensible to lossless. Most new video codecs support lossless coding as well (H.264, H.265, Daala) which is a much more complex area and still achievable. I wonder why audio codecs are not developed with this criteria in mind. Would it reduce CELT's efficiency considerably?

This isn't dead.  I just haven't come back to run my mouth because I don't have anything besides wrapper stuff to show for it, and upon further consideration, that code has to change.  The wrapper code was written to put CELT and lossless data in different packets.  That was before I understood how multistream worked.  Now that idea just seems ridiculous.  Also, there's no way I'm trusting stuff like mkvmerge to say, "Hey, it's a packet full of jibberish with no real Opus data in it.  Yes, I will mux this in."

No, the lossless deltas will be embedded directly in the padding for each packet.  The wrapper around libopus will be written as a proof of concept that I can embed junk in Opus packets and get away with it in a variety of scenarios.  Then after that, I will start on my own CELT implementation and the fun stuff will begin...

There seems to be some sort of decoder state I will have to synthesize when seeking.  That might be a nail.  If I give up on this project, everyone here will know.

Lossless Extensions for Opus (Backwards Compatible)

Reply #86
Okay sorry, it's just there was no update for months, but now i understand.

I don't know by the way which is the better path. Putting correction data into the same container, the same way MPEG4-SLS do by mp4, or into a separate file how WavPack Hybrid does it. The latter makes very easy to copy only the lossy part to somewhere else but the number of files explodes if you have lot of audio on your drive. The first requires a tool to copy only the lossy part.

Lossless Extensions for Opus (Backwards Compatible)

Reply #87
I've already decided on using the same container.  Makes for simpler management.  I had trouble deciding if I wanted the deltas in the same packets as the lossy CELT bits or if I wanted them in their own extension packets.

EDIT:  JMV has expressed concern that I'm creating a rogue extension.  Each padding area will start off with its own identifier like "wswartzendruber_whacked_out_experiment" or some such thing.  If future, standards-compliant decoders that support some new official extension mistake my lossless delta bits for something else, that's on him and the IETF.

EDIT:  Stuffing the lossless deltas in the padding of the CELT packets also absolves some time stamping issues I've been concerned about when it comes to Ogg encapsulation.

Lossless Extensions for Opus (Backwards Compatible)

Reply #88
This stuff died quickly.

Nevertheless i wonder why CELT was not designed in mind to be extensible to lossless. Most new video codecs support lossless coding as well (H.264, H.265, Daala) which is a much more complex area and still achievable.


I wouldn't say that video codecs are more complex.  Since the input data size is is ~ 3 orders of magnitude larger, and the encoding speed isn't exponentially slower, the complexity is probably much lower than a typical audio codec.  Theres just a lot more data to process.

I wonder why audio codecs are not developed with this criteria in mind. Would it reduce CELT's efficiency considerably?


Lossless mode in H.264 basically skips most of the codec and just does lossless packing.  Not quite the same as being extensible to lossless, more like just including two different codecs in one format.  I think from the point of view of CELT, there is a lossless mode, and it is called "FLAC".

Lossless Extensions for Opus (Backwards Compatible)

Reply #89
EDIT:  JMV has expressed concern that I'm creating a rogue extension.  Each padding area will start off with its own identifier like "wswartzendruber_whacked_out_experiment" or some such thing.  If future, standards-compliant decoders that support some new official extension mistake my lossless delta bits for something else, that's on him and the IETF.


So what you're saying is that the IETF is no longer free to have an extension system based on an 8-bit ID because it would make collisions likely (in your example, if an 8-bit ID were "w" there would be a collision). That pretty much fits the definition of a rogue extension.

Lossless Extensions for Opus (Backwards Compatible)

Reply #90
So what you're saying is that the IETF is no longer free to have an extension system based on an 8-bit ID because it would make collisions likely (in your example, if an 8-bit ID were "w" there would be a collision). That pretty much fits the definition of a rogue extension.

What I'm saying is that you've been getting in my way from day one and that's about all you've been doing.  I have approached you, very tactfully, in the Opus mailing list about how to best have an extension.  I recall you telling me to get lost.

Despite what you may think, I really do care about where the IETF is going to take Opus and I do wish to remain compatible.  That is why I approached you.  But if your only response is "get over it" and you guys somehow decide that an 8-bit extension ID without a null terminator is a good idea, then yes, it will be a rogue extension.

Tell me something...  After openly standardizing a royalty-free codec, how did you not realize this would eventually happen?

Lossless Extensions for Opus (Backwards Compatible)

Reply #91
I think people generally expect poorly thought out extensions that break things. There is nothing you can do to completely prevent them and historically a great many have cropped up for popular formats.

The best you can do is discourage them.

Lossless Extensions for Opus (Backwards Compatible)

Reply #92
Tell me something...  After openly standardizing a royalty-free codec, how did you not realize this would eventually happen?


When winter arrives, I realize I'm likely to catch a cold. Doesn't mean I look forward to it.

On a different topic, how about you start by showing us how great you new extension works in practice and then I may take you a little bit more seriously.


 

Lossless Extensions for Opus (Backwards Compatible)

Reply #94
Most new video codecs support lossless coding as well (H.264, H.265, Daala) which is a much more complex area and still achievable.
The adoption and use of scalable coding is almost non-existent. If you count MVC (using just the left image to get 2D, and both to get 3D) then that's one (tenuous) example.

As for scalable to lossless, does anyone have a single example of where a significant number of people actually use this?

I know there are standards for scalable, and scalable to lossless - and I know people play with this stuff - and I know of some situations where it even kind of makes sense (e.g. wavepack lossy) - but even then it's a niche of a niche. Doing it from a psychoacoustic/psycho-visual lossy core makes very little sense and becomes a niche of a niche of a niche.

Don't get me wrong - it's great fun to play with this, and you'll always learn something by experimenting - but please understand there's at least one very good reason why almost no one is using this approach. If you want to solve the problem of lossless/lossy co-existance, there are other things you could do that more people would use - you might even be the first to do them exceptionally well based on Opus. e.g.
http://sysadminsjourney.com/content/2008/1...-mp3-fly-mp3fs/
https://github.com/robinbowes/flac2mp3
or some new idea.

Cheers,
David.

Lossless Extensions for Opus (Backwards Compatible)

Reply #95
If you want to solve the problem of lossless/lossy co-existance, there are other things you could do that more people would use - you might even be the first to do them exceptionally well based on Opus. e.g.
http://sysadminsjourney.com/content/2008/1...-mp3-fly-mp3fs/
https://github.com/robinbowes/flac2mp3
or some new idea.

I don't understand what transcoding software has to do with "scaling"?

Never mind, I misread your post.

Lossless Extensions for Opus (Backwards Compatible)

Reply #96
Well, none of the players I've tested are falling for a Code 3 packet with padding and zero frames.  MPlayer outputs massive warnings and Chrome just goes into cardiac arrest.

EDIT:  That means I have to take the alternative route and embed the deltas in packets after the CELT info.

Lossless Extensions for Opus (Backwards Compatible)

Reply #97
As for scalable to lossless, does anyone have a single example of where a significant number of people actually use this?

Go to your local retail store and find the Blu-rays.  Most of those will have the primary audio track in DTS-HD.

Lossless Extensions for Opus (Backwards Compatible)

Reply #98
Good grief.  Firefox can't even handle packets (made with opus_packet_pad) that have had 64 bytes added to them.  Chrome and GStreamer do just fine handling a 4x padding increase.

Lossless Extensions for Opus (Backwards Compatible)

Reply #99
Apparently editing posts has some kind of expiration.

Anyway, here's a sample file with random garbage for padding.  Each packet has a different amount.  I yanked the source from 2L's free-to-download demo page.

http://www.wswartzendruber.name/uploads/op...g-test-001.opus

I'm interested in finding out about different people's playing experiences with it.