Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Opus killer sample at 256kbps (Read 3923 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Opus killer sample at 256kbps

While testing for opus transparency i found that it does not do well when there is a lot of high frequency noise, so I used audacity to manipulate white noise into the worst case scenario to see how it does and ended up creating a sample that is clearly not transparent even at 256kbps. i used opusenc sample.flac --bitrate 256k 256.opus to encode it.
And also a warning, this is not pleasant to listen to.

Re: Opus killer sample at 256kbps

Reply #1
Yes. Too audible.
Code: [Select]
foo_abx 2.0.6d report
foobar2000 v1.6.9
2022-01-31 21:37:46

File A: sample.flac
SHA1: b8dcec68f3ae6fd0978ea7473eb7276d1447cce3
File B: 256.opus
SHA1: bb6e767326969e472512ab9947a6ecdab7579015

Output:
Default : Primary Sound Driver
Crossfading: NO

21:37:46 : Test started.
21:37:58 : 01/01
21:38:06 : 02/02
21:38:12 : 03/03
21:38:21 : 04/04
21:38:27 : 05/05
21:38:33 : 06/06
21:38:40 : 07/07
21:38:46 : 08/08
21:38:50 : 09/09
21:38:55 : 10/10
21:38:55 : Test finished.

 ----------
Total: 10/10
p-value: 0.001 (0.1%)

 -- signature --
e25c9959978e97da214128286fd181cc6509e2ef

Re: Opus killer sample at 256kbps

Reply #2
A does not imply B

"Bad results on hand-made samples" does NOT imply "encoder is bad on music, speech etc material"

It's easy to break ANY lossy encoder exploiting its weaknesses by crafting some peculiar,  weird, unrealistic (like your) sample.
It doesn't prove that encoder performs bad on a normal material (music, speech etc.)

It was discussed 1000 times in the past. Testing lossy codecs on hand-crafted unrealistic material is useless.

Re: Opus killer sample at 256kbps

Reply #3
What I found interesting is that the png files have more downloads than the audio files :))

Re: Opus killer sample at 256kbps

Reply #4
Sometimes makes you wonder are some people just anti lossy compression, they must find ways to break it.  :)

Re: Opus killer sample at 256kbps

Reply #5
A does not imply B

"Bad results on hand-made samples" does NOT imply "encoder is bad on music, speech etc material"

It's easy to break ANY lossy encoder exploiting its weaknesses by crafting some peculiar,  weird, unrealistic (like your) sample.
It doesn't prove that encoder performs bad on a normal material (music, speech etc.)

It was discussed 1000 times in the past. Testing lossy codecs on hand-crafted unrealistic material is useless.

This isn't the 60s - not every piece of music is created by recording acoustic instruments.

ANY AUDIBLE SOUND CAN BE USED IN MUSIC.

If there is an audible difference between input and output the codec has failed. Tell me, who defines what a 'realistic' sample is? You? Who defines 'normal material'? Again, is it you?

Re: Opus killer sample at 256kbps

Reply #6
There's no way for any properly enginereed sound to contain such strong noise in high freqs. Some PWM 1-bit music comes to mind only, but still it wouldn't be like that. It's kinda like a painting standing next to 20k lumen light pointing straight into your eyes.

Re: Opus killer sample at 256kbps

Reply #7
There's no way for any properly enginereed sound to contain such strong noise in high freqs. Some PWM 1-bit music comes to mind only, but still it wouldn't be like that. It's kinda like a painting standing next to 20k lumen light pointing straight into your eyes.

Ah, I've found him. The arbiter of music (and audio engineering).

I'll append some screens of released, popular music which contains similar signals.

[edit] screens added, all from pressed, commercial CDs.

I mean, what are we even discussing here? If there's an audible difference, the codec has failed.

Re: Opus killer sample at 256kbps

Reply #8
Its not totaly out of the question. I;ve heard similar sounds in
industrial / EBM genres
wavpack -b3.63hhcs.5

Re: Opus killer sample at 256kbps

Reply #9
There's no way for any properly enginereed sound to contain such strong noise in high freqs. Some PWM 1-bit music comes to mind only, but still it wouldn't be like that. It's kinda like a painting standing next to 20k lumen light pointing straight into your eyes.

Ah, I've found him. The arbiter of music (and audio engineering).

I'll append some screens of released, popular music which contains similar signals.

[edit] screens added, all from pressed, commercial CDs.

I mean, what are we even discussing here? If there's an audible difference, the codec has failed.
Does Opus fail with songs you provided SCREENSHOTS for?
If you're looking for a lossy codec that is transparent for every signal in the world you must rethink what you seek, or rather acquire some knowledge how PSYCHO ACOUSTIC based coding works, because you won't find one.
Go encode some 1bit PWM music or oldschool chiptunes with full scale square waves and 100% stereo separation and whine how every psychoacoustic codec fails at mid bitrates. Or how lossless audio codecs fail to compress them better than zip/rar/7z.

Re: Opus killer sample at 256kbps

Reply #10
I'll append some screens of released, popular music which contains similar signals.
Ah yes, thank you so much for the screens of music, because I listen with my eyes. Much appreciated.


edit: But you know, speaking of screens and things we actually do perceive with eyes, imagine this approach to judging lossy video encoders. Arguably, those are even easier to break and with more realistic material than that provided in the OP: just some confetti or a moderate snowfall is enough to bring any lossy video encoder to its knees and render its frames to look like pixel soup. And yet, we accept this as reasonable and don't mind.

Re: Opus killer sample at 256kbps

Reply #11
Ah yes, thank you so much for the screens of music, because I listen with my eyes. Much appreciated.

... Additional to @doccolinni 's words I would like to know which songs these screens were captured from. At least the first one looks like something made especially for purpose of torture.
And, noone will test a lossy movie of image format with unnatural noise pattern and evaluate them with an acoustic transformation of the results. If you want everything lossless then use lossless encoders! This has nothing to to with arbitration.

Re: Opus killer sample at 256kbps

Reply #12
I can't hear the difference between the FLAC and Opus files. Some of these cases tend to show how good the hearing is of the members here. It's transparent to me.

Re: Opus killer sample at 256kbps

Reply #13
I had to raise the volume to hear it. It's a "chirping" artifact at higher frequencies.
But it's not nearly as bad at 150kbps or even at 64kbps. To me, 150 sounds better than 256 here.

Using high bitrates like 256 is not very common for Opus. Maybe that can explain why cases like this slip through...

Re: Opus killer sample at 256kbps

Reply #14
This isn't the 60s - not every piece of music is created by recording acoustic instruments.
You can re-read previous posts and see that nobody says that electronic music can't be or shouldn't be tested on lossy codecs.

If there is an audible difference between input and output the codec has failed. Tell me, who defines what a 'realistic' sample is? You? Who defines 'normal material'? Again, is it you?
It simple. If sample comes from song, speech, CD, radio ... some real source then it's all good.

All formats MP3, AAC, xHE-AAC and Opus have passed formal verification listening tests on samples from real material like songs, speech, ambiental but never non-representative samples like just pure tones of single frequency, sawtooth wave, square wave,  etc.

As you can see it's not just me ... or Hydrogenaudio... 

Its not totaly out of the question. I;ve heard similar sounds in
industrial / EBM genres
It's totally ok to test on industrial/EBM genres or whatever touches your soul and you body.
The discussion is here about a little point of testing  samples with pure mathematical functions which doesn't represent a real use case because ... "Bad results on pure mathematical functions" does NOT imply "encoder is bad on music, speech etc material"

Re: Opus killer sample at 256kbps

Reply #15
Igor, I’m uninterested in the excuses. I have no time for codec sycophants.

The output from the codec is audibly different from the input. You can argue that it’s not a common occurrence, but it remains a problem.

“ It simple. If sample comes from song, speech, CD, radio ... some real source then it's all good.”. Is so meaningless I can’t even comment.

“Pure mathematical functions” is a meaningless attempt at jargon. All audio is a pure mathematical function. Some more complex that others.

What the hell is going on here?! Do people have money invested in these codecs? Is there a fan club I’m unaware of?

I don’t care what the sample is, if it’s easy to ABC at 256kbps, it’s a problem.

Re: Opus killer sample at 256kbps

Reply #16
While testing for opus transparency i found that it does not do well when there is a lot of high frequency noise

May I ask do you have a list of songs or music? Might be interesting to find if there is a problem.

Re: Opus killer sample at 256kbps

Reply #17
The output from the codec is audibly different from the input. You can argue that it’s not a common occurrence, but it remains a problem.
You should seriously review your understanding of lossy coding then.
BTW have you tried to ABX the sample? Is it that bad?  how it compares to another state-of-art codecs?  Are devs willing to make change  for this not representative samples and very likely break the encoder for another actually more popular samples?

Igor, I’m uninterested in the excuses. I have no time for codec sycophants.

I don’t care what the sample is, if it’s easy to ABC at 256kbps, it’s a problem.
We're so sorry to disturb You, your mAJESTY. It will never repeat again.
First of all, nobody is excusing here.  Excusing to whom? To you?  It's just a matter of facts.
Second, there is no "codec sycophants". When You discuss someting you should control yourself and don't attack some individual.

I don’t care what the sample is, if it’s easy to ABC at 256kbps, it’s a problem.
ok, I'm glad that you mentioned this.... now I can tell you without "remorses"  that codec developers (especially Opus ones) have clearly stated that their don't care about corner-cases ( pure noise, pure tones and so on).

So, devs or any other member give a fingers whether You care or not about something!  ;)

“ It simple. If sample comes from song, speech, CD, radio ... some real source then it's all good.”. Is so meaningless I can’t even comment.
So every MPEG codec had a formal testing on real source samples.
It's a fact, not opinion.  What argument You have here to say?

When someone thinks that all people around are wrong and only he/she is right, it's time to look at himself/herself and start to think what's going on.

“ It simple. If sample comes from song, speech, CD, radio ... some real source then it's all good.”. Is so meaningless I can’t even comment.
If You are out of arguments it doesn't mean it's meaningless.  Defend yourself or at least try to do so. Nobody buys your answer  of "dissapointment" here.

Re: Opus killer sample at 256kbps

Reply #18
There's no way for any properly enginereed sound to contain such strong noise in high freqs. Some PWM 1-bit music comes to mind only, but still it wouldn't be like that. It's kinda like a painting standing next to 20k lumen light pointing straight into your eyes.

Ah, I've found him. The arbiter of music (and audio engineering).

I'll append some screens of released, popular music which contains similar signals.

[edit] screens added, all from pressed, commercial CDs.

I mean, what are we even discussing here? If there's an audible difference, the codec has failed.
How about simply post some 30 seconds samples (TOS9) of these tracks as flac so that the rest of us can try them out? Or at least, tell us which CDs and which tracks they are come from.

Re: Opus killer sample at 256kbps

Reply #19
... If there's an audible difference, the codec has failed.
Please review your deep and valuable understandings of lossy compression, how it's tested, how samples are picked up and in which quantity, statistics, interpretations etc.


You can also re-read rutra80 and improve your concept of lossy compression:
Go encode some 1bit PWM music or oldschool chiptunes with full scale square waves and 100% stereo separation and whine how every psychoacoustic codec fails at mid bitrates. Or how lossless audio codecs fail to compress them better than zip/rar/7z.

Re: Opus killer sample at 256kbps

Reply #20
So this is a real world example for those who asked, not music, but it's from 13th episode of an anime called "Serial Experiments Lain", a lot harder to hear but still fails at 256kbps.
Also does anyone know some abx software for linux?


Re: Opus killer sample at 256kbps

Reply #21
The output from the codec is audibly different from the input. You can argue that it’s not a common occurrence, but it remains a problem.

I appreciate the fact that you're completely ignoring my point and only focusing on points you can come up with valid responses to, but let me reiterate:
speaking of screens and things we actually do perceive with eyes, imagine this approach to judging lossy video encoders. Arguably, those are even easier to break and with more realistic material than that provided in the OP: just some confetti or a moderate snowfall is enough to bring any lossy video encoder to its knees and render its frames to look like pixel soup. And yet, we accept this as reasonable and don't mind.


Re: Opus killer sample at 256kbps

Reply #23
So this is a real world example for those who asked, not music, but it's from 13th episode of an anime called "Serial Experiments Lain", a lot harder to hear but still fails at 256kbps.
Also does anyone know some abx software for linux?
My left ear can no longer hear that high frequency, but right ear still can.
Code: [Select]
foo_abx 2.0.6d report
foobar2000 v1.6.9
2022-02-02 15:57:49

File A: sample_2.flac
SHA1: ba9cb8ff3957f23f1243a442b03ba112de2c9839
File B: 256_2.opus
SHA1: 8e17602062d8f7c3460bed141366210c7e7c4872

Output:
Default : Primary Sound Driver
Crossfading: NO

15:57:49 : Test started.
15:58:04 : 01/01
15:58:11 : 02/02
15:58:20 : 03/03
15:58:27 : 04/04
15:58:34 : 05/05
15:58:44 : 06/06
15:58:48 : 07/07
15:58:57 : 08/08
15:59:03 : 09/09
15:59:10 : 10/10
15:59:10 : Test finished.

 ----------
Total: 10/10
p-value: 0.001 (0.1%)

 -- signature --
a0e1f0ccaf7effe52b64fed71e16d85dd1b3f228

Re: Opus killer sample at 256kbps

Reply #24
I can hear it clearly with headphones. There are clear, annoying artifacts on the Opus version. I don't even have trained ears or golden ears, and I'm in my 40s.

Given all this, imo this is relevant. I mean, I listen to Aphex Twin, Autechre, etc, and this could be very well be used in one of their tracks.

Unless this is a well known behaviour and there is no possible fix, it should be looked into.

Quote
foo_abx 2.0.6d report
foobar2000 v1.6.7
2022-02-02 11:00:29

File A: sample.flac
SHA1: b8dcec68f3ae6fd0978ea7473eb7276d1447cce3
File B: 256.opus
SHA1: bb6e767326969e472512ab9947a6ecdab7579015

Output:
Default : Primary Sound Driver
Crossfading: NO

11:00:29 : Test started.
11:01:36 : 01/01
11:02:37 : 02/02
11:02:45 : 03/03
11:03:22 : 04/04
11:03:36 : 05/05
11:03:48 : 06/06
11:03:53 : 06/07
11:03:57 : 07/08
11:04:01 : 08/09
11:04:05 : 09/10
11:04:12 : 10/11
11:04:17 : 11/12
11:04:21 : 12/13
11:04:27 : 13/14
11:04:31 : 14/15
11:04:39 : 15/16
11:04:39 : Test finished.

 ----------
Total: 15/16
p-value: 0.0003 (0.03%)

 -- signature --
ca3d6b261d73d1a124f279896821aaac76a05f4e