Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Replaygain data lost during conversion (Read 9391 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Replaygain data lost during conversion

First of all I would like to thank all those who contributed to the development of fb2k. It's by far the most flexible music-playing application out there imo!

However, I have run into my first problem; the title reveals all.

Applying replaygain manually to the opus files after their generation seems to be the solution for now, presumably because opus' peculiar way of dealing with gain tags has been accommodated for in that department. However, this just adds (seemingly) unnecessary time and effort after the already painfully lengthy process of encoding a vast agglomeration of flacs.

This is all despite the fact that running opusenc on its own in the terminal does retain replaygain data when supplied a flac.

Am I right in thinking that foobar decodes the flac into a plain wav (or something similar) and provides this to opusenc via pipe? It confuses me though that the resulting opus files have all the other tags. Nonetheless, I tried using %s instead of pipe to no avail.

Anyway, could someone tell me if/what I am doing wrong. If this is simply an oversight, would it be possible to patch this for release in a future version?

Thanks in advance for any help!

Replaygain data lost during conversion

Reply #1
Not really a solution to the problem, but in the meantime if the ReplayGain tags are not a necessity and you want foobar to handle the conversion you can try scaling the output of the encoder under the processing option of the converter. It will permanently change the loudness of the opus files according to track or album gain of the original FLAC files, if I'm not mistaken.

No idea what's causing the missing tag issue however, sorry.

Replaygain data lost during conversion

Reply #2
I think this has to do with the album and track peaks being different on lossy encodes.

FLAC


OPUS


Looks like the peaks aren't stored in the Opus tag, just the gain values. is this a bug?
Who are you and how did you get in here ?
I'm a locksmith, I'm a locksmith.

Replaygain data lost during conversion

Reply #3
It seems there are not supposed to be any peak tags as per recommendation from xiph: http://wiki.xiph.org/OggOpus#Comment_Header
Quote
To avoid confusion with multiple normalization schemes, an OpusTags packet SHOULD NOT contain any of the REPLAYGAIN_TRACK_GAIN, REPLAYGAIN_TRACK_PEAK, REPLAYGAIN_ALBUM_GAIN, or REPLAYGAIN_ALBUM_PEAK fields.


Quote
I think this has to do with the album and track peaks being different on lossy encodes.
I think this might be it. Replaygain tags are lost when converting to any other format! There should be an option somewhere to ask if replaygain tags should be kept (even if they are slightly different values on lossless vs lossy), rather than silently destroying them...

Replaygain data lost during conversion

Reply #4
This is not a fault of, or caused by, Opus. The Converter does not transfer ReplayGain data between any formats.

This includes conversions from one lossless format to another, which by definition would not affect average loudness at all. Various members have requested that this ability be added for lossless files, but no reply has been given that I know of.

Replaygain data lost during conversion

Reply #5
Again - this could be allowed as an additional option for those who want it, activated via checkbox in converter window, for example with additional info written in bold, that accuracy of this transfer is not assured. I was also bit confused and disappointed when found this Converter behavior for the first time, but I don't find it as a big problem.

Replaygain data lost during conversion

Reply #6
I'd be for such an option as well. The variation in perceived volume between lossy and lossless shouldn't be that big anyway? Or is it easily perceivable depending on music type?

Replaygain data lost during conversion

Reply #7
No, it isn't. The amount of signal energy lost while lossy transcoding isn't that much - at least when you produce "listenable" lossy files (like at least 192 kbps mp3) that cut off at 19 kHz and don't cut too much in pass band. But if you remove 95% of bandwidth when converting to 1 kbps it can be completely different

Replaygain data lost during conversion

Reply #8
Another vote for tag transfer:
Sometimes one has scanned different parts of an album separately, say if a live album has bonus tracks that are not equally loud. Better transfer tags than giving up that grouping.

Replaygain data lost during conversion

Reply #9
Others can debate lossy formats if they wish. At the very least, I can’t think of any reason to omit to copy the tags between lossless formats.

In case someone mentions that foobar2000 used to use the original ReplayGain algorithm but now uses EBU R128, (i) the switch was made over two years ago, and (ii) users can easily re-scan the files before or after converting, and an automatic transfer would not hinder this ability or otherwise alter the workflow whatsoever.

Replaygain data lost during conversion

Reply #10
Yes, the option of transferring gain tags between lossless formats should be a human right ! I am sure a great many people would be extremely grateful for the option to transfer gain tags between lossless formats !

For lossless->lossy, perhaps a warning message next to the checkbox saying something like "Warning: Gain data will not be 100% accurate after encoding to lossy"? Surely, the user can decide on their own whether to be so intrepid as to go against the aversions of the foobar developers to the transferring gain tags between lossless and lossy; I am sure not too many of the user base will become too bellicose .

Replaygain data lost during conversion

Reply #11
For those utilizing a replaygain option involving "prevent clipping according to peak", it's better that the peak information be accurate.

Replaygain data lost during conversion

Reply #12
For lossless->lossy, perhaps a warning message next to the checkbox saying something like "Warning: Gain data will not be 100% accurate after encoding to lossy"? Surely, the user can decide on their own whether to be so intrepid as to go against the aversions of the foobar developers to the transferring gain tags between lossless and lossy; I am sure not too many of the user base will become too bellicose .
Considering they warn but still allow transcoding lossy -> lossy, I don't see why the same cannot be done for lossless -> lossy replay gain trasfer, which would also serve as a warning in the case mentioned by BenB

Replaygain data lost during conversion

Reply #13
I have always wanted this option for lossless and lossy.

For lossy, an option to only re-scan peak but not replaygain should also be included. It is only the change in peak that is EVER of significance when using good quality lossy encoding (unless you add something to intentionally scale the audio).

Cheers,
David.

Replaygain data lost during conversion

Reply #14
Current alpha version has option to copy ReplayGain data, but peak information is dropped for lossy targets. I think some good arguments are required to change that decision, especially as foobar's ReplayGain scanner is barely slower than pure file decoding.

Replaygain data lost during conversion

Reply #15
Current alpha version has option to copy ReplayGain data, but peak information is dropped for lossy targets. I think some good arguments are required to change that decision, especially as foobar's ReplayGain scanner is barely slower than pure file decoding.
I see your point, and agree.

Anyone know what players do if they find ReplayGain data without peak data? I've never tried that. I've tried wrong data (with usually predictable results), but never no data.

Cheers,
David.

Replaygain data lost during conversion

Reply #16
I have always wanted this option for lossless and lossy.

For lossy, an option to only re-scan peak but not replaygain should also be included. It is only the change in peak that is EVER of significance when using good quality lossy encoding (unless you add something to intentionally scale the audio).

Cheers,
David.

My point of view:
ReplayGain scanner is pretty much always bottlenecked by reading+decoding now. Scanning for peaks alone will be somewhat faster than recalculating peaks+gain - but not much faster and usually HDD bottlenecked anyway.
Microsoft Windows: We can't script here, this is bat country.

Replaygain data lost during conversion

Reply #17
I think some good arguments are required to change that decision, especially as foobar's ReplayGain scanner is barely slower than pure file decoding.

My argument would be convenience, if the user is fine with the transferred peak not being as accurate as re-calculated peak, in turn not having to load files back to foobar and run the scan before transferring to a mobile device (or worse, having to scan from the mobile device), let them.

I see no harm in doing so, especially since foobar is all about options. It could even be a "hidden" tickbox in advanced preferences, but please let us do that (both lossy and lossless transfer)

Replaygain data lost during conversion

Reply #18
ReplayGain scanner is pretty much always bottlenecked by reading+decoding now.


... so, that means that the option to RG scan after conversion is - provided no DSPs are applied - inefficient, as it would take almost no time had the scanning been done while the files were read first time? ;-)

Replaygain data lost during conversion

Reply #19
Current alpha version has option to copy ReplayGain data, but peak information is dropped for lossy targets. I think some good arguments are required to change that decision, especially as foobar's ReplayGain scanner is barely slower than pure file decoding.
After running into issues with rockbox not wanting to apply RG when peak tags are not present, I had a talk with rockbox devs on irc.
In the end I was told that files containing gain, but not peak tags are not conforming with the spec as said here.

That section is rather vague, if the peak tag is needed for volume normalization, then why isn't it listed in the corresponding section? And if it's not, then why does the spec say you need 4 tags?

As far as I see it, you actually only need one tag for volume normalization - either track OR album gain.
Additionally, if the user needs clipping protection, you need the corresponding peak tag - so it's a pair at most, not four of them.

As it stands now, I don't know who is right and who is wrong, since each side seems to think they're in the right.


Replaygain data lost during conversion

Reply #20
If you have a positive ReplayGain value, or a positive pre-amp value applied, or lossy coding that increases the peak, then simplistically you need the peak value to know if you're going to clip or not.

I think the rockbox audio pipeline might have more flexibility than most and be able to handle clipping smartly, but even so, it's reasonable to want the peak value.

It should be able to cope with only track_gain + track_peak, or only album_gain + album_peak being present. It should also fallback to which ever pair is in the tag (if there's only one pair in there) when the user has asked for the other one, rather than not applying ReplayGain at all. e.g. if user asks for album gain, but only track gain is tagged, use track gain, not nothing.


It seems there are not supposed to be any peak tags as per recommendation from xiph: http://wiki.xiph.org/OggOpus#Comment_Header

I didn't read this properly before, and I'm not up to speed on Opus, but unless I'm missing something they're making a mistake here. It's fine to do loudness normalisation based entirely on EBU R128, but it'll work far better if you have peak values, and clearly defined fields for track values, album values, and (if you choose to apply the gain, and that can be done losslessly like in mp3gain) "undo" values. Maybe they're trying to enforce EBU R128 album gain by default, which is a noble aim - but as one audio format in a sea of audio formats, that might be a step too far. I don't know. Is the peak value stored elsewhere? If not, it's going to make life difficult on DAPs.

Cheers,
David.

Replaygain data lost during conversion

Reply #21
If you have a positive ReplayGain value, or a positive pre-amp value applied, or lossy coding that increases the peak, then simplistically you need the peak value to know if you're going to clip or not.
Correct, but what if you don't care about clipping? Then peak should be irrelevant, should it not?

Quote
I think the rockbox audio pipeline might have more flexibility than most and be able to handle clipping smartly, but even so, it's reasonable to want the peak value.

It should be able to cope with only track_gain + track_peak, or only album_gain + album_peak being present. It should also fallback to which ever pair is in the tag (if there's only one pair in there) when the user has asked for the other one, rather than not applying ReplayGain at all. e.g. if user asks for album gain, but only track gain is tagged, use track gain, not nothing.
That's exactly what it does, which makes perfectly sense, the only case where it does not (for me), is that in case there is no peak, imho you should just skip the verification of whether it clips or not, instead of dropping RG completely.

Their argument is that the spec requires you to have a pair of peak+gain to do anything - which doesn't make sense to me in this particular scenario. Basically what I'm asking is whether it really needs peak for pure volume normalization, and if not, then whether the specification text should be changed to clarify that.

Replaygain data lost during conversion

Reply #22
what if you don't care about clipping? Then peak should be irrelevant, should it not?
It's irrelevant to you. I totally take your point, but they might have other concerns. e.g. the rockbox devs might feel it's a useful check of the tag's validity (irrespective of whether you're using the peak data or not), or might have written the code to only work with peak data present.

The original spec didn't anticipate there not being peak data available, and assumed players would use it by default...
http://replaygain.hydrogenaudio.org/proposal/player.html
...and when the idea of copying ReplayGain data from lossless to lossy files first came up, the suggestion was to store the fact it was approximate...
http://www.hydrogenaudio.org/forums/index....showtopic=15445
(item 3) rather than to drop the peak values entirely. That "it's approximate" flag has never been implemented AFAIK, and ReplayGain scanning is a lot faster in 2014 than 2003.

I don't know about Opus, but on any other format, if you've somehow ended up with a collection of files with ReplayGain tags but no peak values, and you don't care about peak values, I think you can drag your entire collection into mp3tag (or even fb2k), add REPLAYGAIN_TRACK_PEAK=1.000000 and REPLAYGAIN_ALBUM_PEAK=1.000000 tags to all your files, and everything will work as you want in your rockboxed player.

I now see the problem with Opus was raised a long time ago...
http://www.hydrogenaudio.org/forums/index....showtopic=97357
https://www.ietf.org/mail-archive/web/codec...t/msg02935.html
...I wish I'd had the time a few years back to take an active role in this. It seems it's too late now? Having to decide before encoding whether to hard-code album gain into the encoding is just wrong. Fine as an option; really poor as the only way a specific audio format allows album-based loudness normalisation! Yet there's a note that ReplayGain is half-complete for Opus...
http://wiki.xiph.org/OPUS_TODO
...have they done it "correctly"? I can't find any other info.

FWIW the way some ReplayGain tags are now written with a -23LUFS reference level is also a mistake I wish I'd prevented. Oh well. People got hung up on the original calculation (which was always intended to be improved), rather than the point...
http://www.hydrogenaudio.org/forums/index....mp;#entry736023

Cheers,
David.
P.S. sorry for derailing this thread. I should have searched for the others earlier.

 

Replaygain data lost during conversion

Reply #23
It's irrelevant to you.
Or anyone who only wants to have volume normalization only... but I get what you are saying.

Quote
I don't know about Opus, but on any other format, if you've somehow ended up with a collection of files with ReplayGain tags but no peak values, and you don't care about peak values, I think you can drag your entire collection into mp3tag (or even fb2k), add REPLAYGAIN_TRACK_PEAK=1.000000 and REPLAYGAIN_ALBUM_PEAK=1.000000 tags to all your files, and everything will work as you want in your rockboxed player.
The main concern here is that applying tags on a relatively slow SD card, or even slower portable player memory is something that takes about the same amount of time as transcoding itself. 

"One-click-convert" solution instead of having to rescan or add tags manually is what I'd ideally want to have. Else I might as well just continue copying the FLAC files over, which is a "one-click-copy" solution which takes about the same amount of time (few secs longer) but doesn't involve any interaction from me. Applying track/album gain on encode is less than ideal, because then you can't have a "track gain when shuffling, album gain otherwise"-type of flexibility anymore.

On a slightly different note, I thought about suggesting calculating peak tags on-the-fly, but that would only work for track peak, as you can't reliably detect which tracks belong to an album and whether the album is complete or not, so even going in that direction wouldn't change much. And switching to album gain would be impossible because peak is still missing for it.

What about an option to "add custom tags" in the converter? That's where people who don't care about peak could just make a preset which sets the peak to 1.0 and move on.