Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency? (Read 2490 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Hello,
    to test encoding at some very low bitrates, I encoded a sine sweep at 12 kbps with Opusenc, set the "--expect-loss" to a value of 90, and then decoded the resulting file with Opusdec. The output seemed to be very similar to choosing a bitrate of, let's say, 10 kbps while leaving "--expect-loss" at 0.
    The strange thing was that even though the output wave file was at 48 kHz, it showed clear marks of being resampled from 16 kHz and where my original frequency reached
16 kHz, the frequency in the output file practically fell down to zero.
    Unfortunately, it seems that the lowpass filtering is somehow a part of the actual resampling algorithm. Why do I think so? Opus normally doesn't encode
any frequencies above 20 kHz, which would suggest that 20 kHz should be aliased down to 4 kHz followed by silence. However, right at the spot where my
original sine sweep went all the way up to almost 24 kHz, the aliased frequency reached almost 8 instead of being absent altogether.
    Now I can hear someone say that natural sounds are not composed solely of some very high frequencies. But I still think that aliasing is an undesirable
effect. After all, when the final sample rate is supposed to be 48 kHz anyway, why should low bitrates make it necessary to first downsample the audio
and then upsample again?

Re: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Reply #1 –
I'm not sure what you did exactly or why, but when encoding at 12 kb/s, the encoder basically decides that:
1) Trying to encode everything at 48 kHz is not going to produce anything useful
2) Unless told otherwise, it's best to assume the content is speech or similar and encode with SILK rather than CELT. The reasoning here is that if it's speech, then you'll get good results with SILK but not with CELT. And if it's music, it'll be terrible no matter what (slightly worse with SILK, but at that point it's useless anyway).
So based on that, the signal likely went down to 16 kHz for SILK encoding, then back up to 48 kHz. In all cases, any resamplers that actually end up getting used in Opus or its tools will introduce less artifacts than the loss of the codec at the bitrate you're using.

Re: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Reply #2 –
Quote from: jmvalin on
So based on that, the signal likely went down to 16 kHz for SILK encoding, then back up to 48 kHz. In all cases, any resamplers that actually end up getting used in Opus or its tools will introduce less artifacts than the loss of the codec at the bitrate you're using.
Thanks.
I understand that. But still, in certain specific situations, the issues which seem to be caused by some poor anti-aliasing filters are audible. If my uncompressed recording includes a tone that glides from 12 to 14 kHz and after encoding and decoding it goes from 4 kHz down to 2, then I'm unable to find any other explanation for it than the fact that the filtering used in the resampling algorithm is too weak. The reason why I'm talking about it is that if the sound were properly filtered before resampling to 16 kHz, then the encoder would not have to actually deal with the aliased frequencies in the first place. And as I've said, if I either set "expec-loss" to a higher value or if I set the bitrate even lower than 12, the CELT part vanishes and I can clearly hear that there's been aliasing introduced into the SILK part.

Re: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Reply #3 –
Quote from: harmonion on
If my uncompressed recording includes a tone that glides from 12 to 14 kHz and after encoding and decoding it goes from 4 kHz down to 2, then I'm unable to find any other explanation for it than the fact that the filtering used in the resampling algorithm is too weak.

IIRC there are a couple different resampling options in Opus, you could play around with them and see if its specific to one of them, but in general you're probably not going to find extremely high image rejection in the resampling used in an audio codec for performance reasons.  If you've got some particular case where your signal is contaminated by a strong out of band tone, you'll probably want to preprocess your audio first rather than rely on the encoder to handle it. 

Re: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Reply #4 –
Quote from: saratoga on
IIRC there are a couple different resampling options in Opus, you could play around with them and see if its specific to one of them, but in general you're probably not going to find extremely high image rejection in the resampling used in an audio codec for performance reasons.

Are there really? Where do I find any info on how they're used?

Quote from: saratoga on
If you've got some particular case where your signal is contaminated by a strong out of band tone, you'll probably want to preprocess your audio first rather than rely on the encoder to handle it. 

Encoders for lossy formats usually resample the audio when the bitrate is very low. For MP3 or AAC, the corresponding decoders usually give the result at this lower effective sample rate. Opus is different because the encoder stores the information about the original sample rate even when the bitrate is awfully low. Trouble is, most users won't know if the encoder decides to use "Superwideband" or "Wideband" at the particular bitrate. So if Opusdec decodes the Opus file and then resamples the data to 48 kHz just because the uncompressed file was sampled at 48 kHz, then you can hardly tell what intermediate sample rate the encoder has used internally. So then you can hardly guess which frequencies are out of band.

Re: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Reply #5 –
Quote from: harmonion on
Quote from: saratoga on
IIRC there are a couple different resampling options in Opus, you could play around with them and see if its specific to one of them, but in general you're probably not going to find extremely high image rejection in the resampling used in an audio codec for performance reasons.

Are there really? Where do I find any info on how they're used?

It has been ten years since I looked at this, but I believe it is here:  https://github.com/xiph/opus-tools/blob/master/src/resample.c

Quote from: harmonion on
Trouble is, most users won't know if the encoder decides to use "Superwideband" or "Wideband" at the particular bitrate. So if Opusdec decodes the Opus file and then resamples the data to 48 kHz just because the uncompressed file was sampled at 48 kHz, then you can hardly tell what intermediate sample rate the encoder has used internally. So then you can hardly guess which frequencies are out of band.

Does this actually happen with real audio?  If not, then no problem for regular users, just testing with pure tones which tend to cause problems for lossy codecs. 

Re: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Reply #6 –
Quote from: harmonion on
I understand that. But still, in certain specific situations, the issues which seem to be caused by some poor anti-aliasing filters are audible. If my uncompressed recording includes a tone that glides from 12 to 14 kHz and after encoding and decoding it goes from 4 kHz down to 2, then I'm unable to find any other explanation for it than the fact that the filtering used in the resampling algorithm is too weak. The reason why I'm talking about it is that if the sound were properly filtered before resampling to 16 kHz, then the encoder would not have to actually deal with the aliased frequencies in the first place. And as I've said, if I either set "expec-loss" to a higher value or if I set the bitrate even lower than 12, the CELT part vanishes and I can clearly hear that there's been aliasing introduced into the SILK part.

The SILK resampler is designed a bit differently from regular resamplers. Because SILK targets voice, its resampler actually allows a bit of aliasing and instead maximizes the bandwidth. The reasoning is that the spectrum of speech decreases monotonically and tends to be quite noisy as frequency goes up. Therefore, a bit of 9 kHz noise being aliased to 7 kHz at a much lower level won't actually be audible. There's no realistic situation where the SILK aliasing would be audible for speech.

Re: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Reply #7 –
Quote from: jmvalin on
The SILK resampler is designed a bit differently from regular resamplers. Because SILK targets voice, its resampler actually allows a bit of aliasing and instead maximizes the bandwidth. The reasoning is that the spectrum of speech decreases monotonically and tends to be quite noisy as frequency goes up. Therefore, a bit of 9 kHz noise being aliased to 7 kHz at a much lower level won't actually be audible. There's no realistic situation where the SILK aliasing would be audible for speech.
Thanks. That makes some sense.
#1. Is the choice of "Wideband/Superwideband/Fullband etc." entirely up to Opusenc or is the user expected to have some control over this? I'm asking primarily to know if it is somehow related to the "--rate" switch in Opusdec.
#2. Am I right in assuming that in Fullband the entire spectrum is encoded with CELT instead of encoding the lower part with the modified SILK?
 

Re: Internal resampling in Opusenc; has anyone tested its anti-aliasing efficiency?

Reply #8 –
Quote from: harmonion on
#1. Is the choice of "Wideband/Superwideband/Fullband etc." entirely up to Opusenc or is the user expected to have some control over this? I'm asking primarily to know if it is somehow related to the "--rate" switch in Opusdec.
It's more complicated. opusenc will take a file sampled at a certain sampling rate, so it will tell Opus not to encode it with more bandwidth than the file had to begin with. But even if the file is (e.g.) 48 kHz, if the encoder is able to tell that there's no frequencies above 8 kHz, it may still use wideband (not all the time because sometimes it's not safe). On top of that, if the bitrate is too low, Opus may still decide to reduce the bandwidth to avoid causing too many artifacts (no point in trying to encode fullband at 9 kb/s).

Quote
#2. Am I right in assuming that in Fullband the entire spectrum is encoded with CELT instead of encoding the lower part with the modified SILK?
SILK only goes as high as wideband, so for fullband CELT has to be used, but they can sometimes be used together (hybrid mode). For fullband speech below about 64 kb/s (not an exact threshold) SILK will be used to code 0-8 kHz and CELT will be used for 8-20 kHz. For music or when the bitrate is higher, only CELT is used.