Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Opus and replay gain (Read 36057 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Opus and replay gain

The most recent discussion of this is from 2012 so I thought I'd start a new thread.  You can read the old one if you need more background:
http://www.hydrogenaudio.org/forums/index....showtopic=97357

I have implemented an Opus plugin, including support for writing metadata (wish the libraries supported that!), and reading it obviously.  I am still at a loss for exactly how best to deal with gain adjustments.  Although the spec doesn't like the commonly used REPLAYGAIN* comment tags, they seem very likely to occur in Opus files because of the incomplete support for track and album gain and peak values, plus the lack of any tools to supply the gains from the spec.  So if I find REPLAYGAIN* comment tags then I use them.

This leaves some issues.  If there are Opus-spec gains also then these should probably take precedence.  In fact the header gain is applied automatically unless I explicitly override it.  However, I suspect in a music player this isn't going to please some people, at least until album gain (and perhaps peak levels) are widely and consistently implemented in Opus tracks.  Am I going to need configuration options so people can pick what sort of tags they want to use for their gain adjustment?

Opus and replay gain

Reply #1
I think if this is as good as it gets...
http://wiki.xiph.org/OggOpus#Comment_Header
...then it's a mess. I already commented here...
http://www.hydrogenaudio.org/forums/index....st&p=856065

There are two flawed ideas:
1. Album gain should be stored in an almost-always-applied gain field that's not called album gain, and therefore sometimes isn't album gain, but is the default way of hard coding a single gain change for that file.
2. You don't need to know the peak amplitude.


I don't know enough about Opus to be sure, but my hunch is to do the following to clear the situation up:

1. Leave output gain alone and separate from ReplayGain. Use it like the frame-level Global Gain field in mp3. Therefore:
2. Always apply the Output Gain
3. Any other gain correction tags (including ReplayGain), should be calculated after the Output Gain is applied, and applied after (or with) the Output Gain.

Follow those rule, and you've simplified things dramatically.

Then it's easy:
R128_TRACK_GAIN = REPLAYGAIN_TRACK_GAIN - 5
i.e. you have those two tags effectively carrying the same data (but in different formats). Write them both. Read either one. If there are ReplayGain tags, I think they should be used in preference to R128_TRACK_GAIN (but then I would say that  ) because they carry the full set of information you need. Only fall back to R128_TRACK_GAIN if you have to.

The other three ReplayGain tags carry unique data (in the normal way) - album gain, track+album peak.
The peak values should be the peak after the Output Gain is applied. I know different decoders give a different result. It's still better than having no information at all.


Don't store -23LUFs gains in ReplayGain tags. Do it the way fb2k does it: use the EBU R128 calculation, but use a -18LUFs target, which matches the level used in ReplayGain tags for the last ten years. If people want -23LUFs on playback from ReplayGain, set the RG pre-amp to -5. Job done.


This doesn't break the Opus spec. If someone chooses to put the album gain into Output Gain, it'll just look to all other tools as if the audio is already EBU R128 normalised by album (just like mp3s that have had album gain applied in mp3gain). Running a ReplayGain scanner won't break this.

Cheers,
David.

 

Opus and replay gain

Reply #2
Have you asked in #opus ?

Opus and replay gain

Reply #3
I still don't understand why the replaygain-field stores the difference to a reference volume. Wouldn't it be a lot easier if just the volume of a track and the volume of the whole album would be stored? Like this it's somehow a mess...

IMHO it should be the player to decide then which reference volume is used.

Opus and replay gain

Reply #4
I still don't understand why the replaygain-field stores the difference to a reference volume. Wouldn't it be a lot easier if just the volume of a track and the volume of the whole album would be stored?

because Human loudness perception depends on absolute loudness and frequency. The Human "Equal Loudness" contours aren't straight or equally spaced at most frequencies. Crank the volume control up by 20dB, and the perceived loudness change might be 25dB for one track, 15dB for another track (though those are extreme examples).

ReplayGain was designed so that the original algorithm could be replaced to get better loudness matching, while keeping the tags and players the same.

If someone invented an artificial loudness measurement that matched human ears perfectly, what it could tell you is how much to change the gain on a given track to hit a given loudness. It couldn't perfectly tell you how loud something was unless you told it how loud you were listening, and that's less useful, because as soon as you apply a gain change to hit some wanted level, you're not listening at the loudness level the calculation used.


Let me put it another way. You're assuming that if you want to listen at 83dB, and a tag tells you that the track is 63dB loud (using some measurement), you need a 20dB boost. That's a pretty good estimate generally, but it could be several dB off - tens of dB wrong if the track was really bass heavy. A player can't know what it should really do. Whereas if the tag tells you that, to listen at 83dB, you need to boost it by 15dB, the player knows exactly what to do.


Neither the original ReplayGain, nor EBU R128, take account of the varying equal loudness contours at different loudness level properly, so for both these algorithms you'd always get the same result whether you stored "it's x dB loud" or "it needs to be made y dB louder". But if someone did write an algorithm that took account of the varying equal loudness contours at different loudness levels, only the second approach would work - which is why ReplayGain tags work that way.

Cheers,
David.

Opus and replay gain

Reply #5
Back to the original question

I have been to #opus, but on a different matter.  Fastest ever bug fix I've seen!

I haven't discussed replay gain with Xiph, although plenty of others have.  Their position is clear, but not especially well suited to the situation with music players.  2Bdecided has summarised the main issues, but I'll describe some specific situations I'm trying to address.

1. Opus header gain may or may not be album gain on a specific track.  If it is being used for that purpose and my player is configured to ignore replay gain then I should not apply the header gain.  Even though the Opus spec says it should always be applied, it is clearly up to the listener whether they want album gain (or track gain) to be used at any particular time.  However, if it is not an album gain then I should apply it in accordance with the spec.  So I've already broken 2Bdecided's rule #2 (and implicitly rule #1).

2. If there are separate REPLAYGAIN* tags then I feel they should be used.  If the listener doesn't want to use them then they shouldn't have put them in there.  However it isn't at all clear that they will have been calculated wrt to levels adjusted by any existing header gain.  Maybe they are, maybe they aren't. The spec clearly has nothing to say except don't do it because it will be confusing.  So I don't feel that I can rely on rule #3.

I'm not actually calculating replay gain, so I don't have to decide which tags to create.  I just copy over whatever I'm given.  I can assume the three rules as a default position, but I hope I've explained reasons why I want to allow options for different behaviour.  The ability to not apply even the header gain in some situations I feel is essential, whatever the spec might say (if Xiph wanted a header gain that could always be unambiguously used then they should have supplied a separate album gain that could be ignored).  I also think that the ability to treat REPLAYGAIN* gains without the header gain is also important, because of the lack of any standardised behaviour (or any behaviour at all!) at this point and just because the results of getting it wrong could be so bad.

Opus and replay gain

Reply #6
I haven't discussed replay gain with Xiph, although plenty of others have.
Are there any links, or were these just on-line chats?

Quote
Their position is clear, but not especially well suited to the situation with music players.
There was some confusion surrounding Ogg Vorbis vs ReplayGain in the early days. It was solved by Garf creating a ReplayGain tool that worked, and everyone else followed the tag format it used. Hence it was Garf, rather than Xiph, that created the ReplayGain Vorbis comment tags that are used to this day.

Cheers,
David.

Opus and replay gain

Reply #7
This should give you the gist of it.  I'm sure you'll be familiar with the arguments from both sides.
https://www.ietf.org/mail-archive/web/codec...eads.html#02935

At the moment I'm looking at my plugin offering two checkboxes:
1. Treat Opus header gain as album gain (default no?) - yes would mean only apply the header gain if the player is configured to use album gain (and no REPLAYGAIN* tags or configured not to prefer them), no would mean apply the header gain to everything, combined with other gains if necessary.
2. REPLAYGAIN* tags override Opus gains (default yes) - no would mean REPLAYGAIN* tags were ignored when Opus track gain (or non-zero header gain being treated as an album gain?) is present.

The default settings mean that Opus header gain is always applied (even if the player is set not to use replay gain), Opus track gain is applied when track gain is configured in the player and there are no REPLAYGAIN* tags, and the appropriate REPLAYGAIN* tag level is combined with the header gain if present.  Other settings allow switching off the header gain and using Opus gain tags even if REPLAYGAIN* tags are present,

I'm still wondering what to do about the level difference between the two types of tag.  It is hardly meeting the goal of replay gain if one track/album plays 5-6dB louder than the next because they were tagged under different standards.  The alternative is to nudge one of them, and it would have to be the Opus gains that were changed, which leaves me breaking things for people who actually want to play something according to the spec.

Opus and replay gain

Reply #8
I'm still concerned about the different reference levels for the two tagging systems.  Of course the problem has been around ever since replaygain moved from 83dB to 89dB, but it was rare for people to have tags from both systems in their files at the same time.  With Opus it is (or will be) perhaps less rare.  In testing with mixed sets of tags, the gain jump between tracks or between albums can be jarring and certainly defeats the purpose of having the tags in the first place.

I don't want to just override the meaning of the tags and adjust them all to the same target loudness, but I'd like to give the user the chance to do that if they end up with mixed tags and consequently non-levelled loudness.  One way would be to let them have an override target loudness.  By default, I'd just let the tags do what they say, but if the user insists then I'd adjust them from their explicit or implicit target loudness to the one specified by the user.

Does this make sense?  I know it is a bit overkill but these are settings that the vast majority if users won't need and won't ever have to look at.

Opus and replay gain

Reply #9
This should give you the gist of it.  I'm sure you'll be familiar with the arguments from both sides.
https://www.ietf.org/mail-archive/web/codec...eads.html#02935
Sorry, you're right, I have seen this before, but forgotten it.

As for the rest, I was trying to see what other software deals with Opus and RG to see if any useful precedent has been set.

fb2k is supporting it, and bumps the R128_TRACK_GAIN up by 5dB before use, to make it match everything else. Not much point having volume-matched files playing 5dB quieter than everything else in a multi-format player just because they happen to be in a different format.

I think you have to make them match, and then give the user the choice to offset the result, which they can do with a pre-amp if they want. Or give them a Reference level switch: ReplayGain 89dB or EBU R128 -23LUFs ("5dB quieter").

I asked Peter (fb2k) what he was doing with/for peak information, but he hasn't replied yet.

Cheers,
David.

Opus and replay gain

Reply #10
I spotted another problem today. It'll be vanishingly rare, but worth mentioning.

If the Output Gain is used to store the ReplayGain Album Gain (calculated to EBU R128 at -23LUFs), and you have a very dynamic album (i.e. one that peaks near digital full scale, but has a measured loudness lower than -23LUFs, hence a positive Output Gain), then you are telling every Opus player to clip it by default.

Seems wrong, but that's the way it is.

Cheers,
David.

Opus and replay gain

Reply #11
I spotted another problem today. It'll be vanishingly rare, but worth mentioning.

If the Output Gain is used to store the ReplayGain Album Gain (calculated to EBU R128 at -23LUFs), and you have a very dynamic album (i.e. one that peaks near digital full scale, but has a measured loudness lower than -23LUFs, hence a positive Output Gain), then you are telling every Opus player to clip it by default.

Seems wrong, but that's the way it is.

Cheers,
David.

Inevitable with that standard, isn't it?  Hence good players offer the choice of peak scaling or not with replay gain.

Opus and replay gain

Reply #12
Yes - but also good formats
1. don't confuse the ReplayGain info with something else
2. include peak information in the ReplayGain tags, and
3. good implementations give users the option to use or not use ReplayGain during playback.

Try to set (never mind use!) a ReplayGain album tag in accordance with the Opus specification, and all three of these things goes wrong (by default).

Cheers,
David.

Opus and replay gain

Reply #13
The design rationale of the Opus design was clearly to make at least album gain just work even if the player developer had never heard of ReplayGain. This completely dwarfs your (1), (2) and (3) which are all based on the assumptions (s)he knows exactly how it's supposed to work, and even has the UI available to let the user make sane choices based on it. In the case of hardware players, that's obviously not true - how many do you know with full ReplayGain configuration?

Conversely, in the current specification, it will just work. If your player knows about ReplayGain, you get track gain too.

The point of clipping is true, but that's a problem even IF you know the peak information. If you know that playing at the right loudness will cause clipping, there's no right way to play back the music. FWIW the current Opus libraries contain helper functions to do soft clipping, because clipping is an issue in DAP even if ReplayGain isn't in the picture at all.

I understand what you want. You want a perfect world. We can get some of that if we for example use Vorbis in foobar2000 or selected DAP, but go outside that much and it's a wasteland, and your Vorbis files turn into a pain to use. Opus' version of ReplayGain was designed so it would work everywhere, at the cost of dropping a feature whose benefit is very questionable (peak info).

Opus and replay gain

Reply #14
Although the spec doesn't like the commonly used REPLAYGAIN* comment tags, they seem very likely to occur in Opus files because of the incomplete support for track and album gain and peak values


The spec is very clear here:
To avoid confusion with multiple normalization schemes, an OpusTags packet SHOULD NOT contain any of the REPLAYGAIN_TRACK_GAIN, REPLAYGAIN_TRACK_PEAK, REPLAYGAIN_ALBUM_GAIN, or REPLAYGAIN_ALBUM_PEAK fields.

This isn't "doesn't like", this is saying it's not a legal at all. If you encounter a file with such tags, that file is corrupted and should be treated as such. How can you treat something "correctly" when it's mere presence is a bug? I recommend not playing them at all so users who are unfortunate enough to use a tool that corrupts their files would notice instantly and can get rid of it.

If you really want to support everything, make it a configurable option whether to apply those gains in addition to or in replacement of correct gain tags. Reminds me a bit of Winamp & foobar2000 having a "broken channel mapping" option when some less-than-attentive developers make Vorbis encodes with the Windows WAV channel mapping instead of the one actually specified in the Vorbis standard.

Opus and replay gain

Reply #15
If you like your clipping you can keep your clipping.

Opus and replay gain

Reply #16
If you like your clipping you can keep your clipping.


Thank you for your enormously constructive argument, which makes it instantly clear what exactly your problem is and allows us to formulate a solution to address it.

Opus and replay gain

Reply #17
I already logged out after my comment when a thought struck me. There is no humor here. I need to go back asap. Ofcourse I find that lo and behold there really isn't.

A solution? Use a volume control. I would.

Opus and replay gain

Reply #18
The design rationale of the Opus design was clearly to make at least album gain just work even if the player developer had never heard of ReplayGain.
Opus is an audio codec.

If you want to see an audio codec built from the ground up to handle loudness normalisation properly, look at AC-3 / Dolby Digital. That works properly when AC-3 is the only audio format in use. This is the de facto position on DVDs, because commercial use of the other two allowed audio formats is almost non-existent.

Once you have other audio formats, it gets quite complicated, and decoders have to follow strict rules to avoid loudness chaos. e.g. see Dolby technical bulletin 11.

Even with these more complicated decoder rules, the content producers have to follow even more complicated rules to make everything work properly. They have to care about dynamic range with and without downmixing. They have to care about clipping prevention at different output levels. They have to care that these parameters are different depending on the audio format in use.

Even broadcasts with multi-billion dollar budgets look at this problem as say "it's too hard - just pass feature film soundtracks through as-is, and produce everything else to EBU R128 -23 LUFs with automated peak compression on downmixes".


Back to Opus, which allegedly is designed from the ground up to support loudness normalisation even if the player developer hasn't heard of it, and clearly exists in a world with other audio formats. It's also going to be encoded by normal people. You cannot expect broadcaster-competent levels of dynamic range management in the source.

In Opus the output gain may, or may not be, the Album Gain.

If the output gain is the Album Gain, but the player developer and/or the user doesn't understand the significance of this, then Opus files will play back about 15dB quieter than mp3s (both ripped from the same 2014 CD), and there's nothing the user can do about it.  How soon before someone writes a tool to batch re-set the output gain on a bunch of Opus files to 0, or creates a player that ignores the output gain by default (hence breaking the Opus spec)?

If the output gain is the Album Gain, but the player developer does understand the significance of this, they can try and improve the experience. This is hampered somewhat by the problem that a) they don't know if the output gain really is the Album Gain (think what happens when it isn't), and b) they don't have a peak level to check when (for example) increasing the loudness above -23LUFs to match other audio.

If the output gain is not the Album Gain, then at least the Opus files will play at the same loudness as mp3s. Though if a user has some Opus files where it is, then those are going to sound very quiet in comparison. By default.

Dolby AC-3 can force loudness normalisation on everyone, partly because the files are professionally and consistently created, partly because their codec sits as the only codec in some situations, and partly because they write very careful rules of how to work with other codecs. Opus cannot tick any of these boxes. Hence Opus cannot force loudness normalisation on everyone.


Quote
This completely dwarfs your (1), (2) and (3)
It would if it did what you say, but as I've shown above, it doesn't.


I mentioned AC-3 as an example of how you design something in isolation from the ground up to support loudness normalisation. If you don't have that luxury - e.g. you are adding something onto an existing format, or designing a format that must co-exist with other formats, then you need to do more, not less. You either do something like Dolby technical bulletin 11, or you write some tags (e.g. loudness and peak tags to a certain reference) which players can use even if the calculation method or audio format changes.

Quote
The point of clipping is true, but that's a problem even IF you know the peak information. If you know that playing at the right loudness will cause clipping, there's no right way to play back the music.
You know there are two ways to mitigate this.
Quote
FWIW the current Opus libraries contain helper functions to do soft clipping, because clipping is an issue in DAP even if ReplayGain isn't in the picture at all.
True, but without peak information, you are only left with the compression/soft-/hard-clipping options, and can never have the "just play it a bit quieter but without damaging the audio" option.

Quote
I understand what you want. You want a perfect world.
I like things to improve. I like thing to be easy to use and difficult to break. I'm sure the Opus developers thought they were delivering the "easy to use" part, but I don't think they have, and they seem to have compromised the other two.

Quote
We can get some of that if we for example use Vorbis in foobar2000 or selected DAP, but go outside that much and it's a wasteland, and your Vorbis files turn into a pain to use. Opus' version of ReplayGain was designed so it would work everywhere, at the cost of dropping a feature whose benefit is very questionable (peak info).
This makes no sense. If you called the Output Gain "Album gain", specified that it was either absent or correctly filled by the encoder, specified that every player must apply it by default (they can add a user option to disable it if they want), and also included peak information - then you'd have exactly the benefits that you seek, but avoid most of the drawbacks. Specifically, not including peak information isn't a required step to get to where you seem to want to. It's not a necessary cost. It's a pointless mistake.

Cheers,
David.

Opus and replay gain

Reply #19
Meanwhile, there is still no opusgain tool 

Opus and replay gain

Reply #20
Meanwhile, there is still no opusgain tool 

One reason why I feel I have to offer support for REPLAYGAIN* tags in Opus files.  Until there is a tool to set real Opus gains, and maybe even after, people are going to slap in whatever gain tags they have whenever they transcode a file.

Opus and replay gain

Reply #21
Well, there's foobar2000.



Opus and replay gain

Reply #24
Meanwhile, there is still no opusgain tool 

One reason why I feel I have to offer support for REPLAYGAIN* tags in Opus files.  Until there is a tool to set real Opus gains, and maybe even after, people are going to slap in whatever gain tags they have whenever they transcode a file.


But are there tools that set REPLAYGAIN tags on Opus files? vorbisgain sure doesn't AFAIK. What idiot would write such a tool? It's almost intentionally malicious to do so. Users manually adding useless tags? Would be pretty strange as well.

So why on earth would you support this?

The only tool that supports ReplayGain scanning on Opus does it correct. So why would you intentionally add support for something the spec specifically says is forbidden? At worst you're now *encouraging* people to make broken files and you've just made your own problem worse.