HI! This is my first post, so let me introduce myself first: I'm Ana and I'm a huge enthusiast of audio technology, and state-of-the-art audio and video. Opus... well, what can I say. I'm just in love with this codec. It's a just magical, there's nothing quite like this. At least, that's what I thought before reaching this reddit comment (https://www.reddit.com/r/Android/comments/9pjy22/opus_13_released/e8686gp/?context=8&depth=9). This guy, apparently (he has actually posted some ABX test), is able to discern Opus @320kbps vs. FLAC, and even vs. AAC, which he believes is superior for this particular tune. He claimed later (in a MP) the difference is easy to spot during 2.08 - 2.15. I just can't. 96kbps is transparent to me. He even says later on that Opus @140kbps sounds like trash. My question is, what do you guys think? Is this a killer sample? Is there some king of bug in the encoder? Does he have those alleged golden ears of which there are no proof of existance AFAIK?
Here is the link to the files (https://onedrive.live.com/?authkey=%21AJrE42FsVClawzs&id=8A05405DF49FFF62%2199163&cid=8A05405DF49FFF62), so that you can hear it by yourselves. Tell me what do you think.
First off: There's a significant misconception in that threat, which is that the codec with less differences to the original is the one with the better quality. If a lossy codec throws away more - to the human ear - inaudible stuff in order to preserve audible segments at higher bitrate it is the better lossy codec, not worse. Arguing from such a position basically ignores the entirety of psychoacoustics.
I've tried my hand at ABXing the tracks from that thread. For a second I imagined to be able to hear a difference, but ABX trials quickly proved me wrong. IMHO if there's an audible difference it's so minute that the smallest lack in concentration makes it go away. But admittedly I'm not super good at ABXing. It's also possible that the guy from the reddit thread has some kind of hearing anomaly that allows him to pick up things that we'd miss, because the psychoaccustics are not tuned for his weird ear.
This kind of "spot the difference" game isn't that interesting to me anyway, particularly when it all comes down to listening to the same 2 seconds of a track 50 or more times until you can make out a tiny difference. At that point it's already way, way, beyond what I'd want out of a lossless codec in terms of quality and I'm more than satisified. I'm not going to care about that when I'm just listening to the music. The problem with that particular track for example is not the lossy encoding, but the loudness and low dynamic range of 5 (according to DR meter), which I can do nothing about... codec barely matters here if everything is already so garbled. I'd have a hard time distinguishing all that clipping (see screenshots) from encoding artifacts.
Thanks for your detailed answer. I agree with you on all the points. I also didn't pass the flac through Audacity (doing it just didn't come to my mind), but looking at it, I wonder where does the FLAC come from. Is that something that would be acceptable for a studio recording? It seems to me the file has been manipulated, may it be?
Is that something that would be acceptable for a studio recording?
Oh you sweet summer child... lol. Yeah, that's pretty normal for any modern metal mastering. And I say this as a pretty big metal fan myself - I have just resigned myself that it will usually (there are some rare exceptions) be crap in terms of loudness, but don't get me started on a rant.
The only thing "odd" I noticed about the flac is that it is 48khz, which means that it may have been upsampled.
reddit comment (https://www.reddit.com/r/Android/comments/9pjy22/opus_13_released/e8686gp/?context=8&depth=9). This guy, apparently (he has actually posted some ABX test), is able to discern Opus @320kbps vs. FLAC
The link and file appears to indicate 256k
reddit comment (https://www.reddit.com/r/Android/comments/9pjy22/opus_13_released/e8686gp/?context=8&depth=9). This guy, apparently (he has actually posted some ABX test), is able to discern Opus @320kbps vs. FLAC
The link and file appears to indicate 256k
Yeah, but later on he went further to say he was able to discern it at 320kbps.
Is that something that would be acceptable for a studio recording?
Oh you sweet summer child... lol. Yeah, that's pretty normal for any modern metal mastering. And I say this as a pretty big metal fan myself - I have just resigned myself that it will usually (there are some rare exceptions) be crap in terms of loudness, but don't get me started on a rant.
The only thing "odd" I noticed about the flac is that it is 48khz, which means that it may have been upsampled.
Weird? May the FLAC have been manipulated? Should I grab a Flac myself?
I tend to agree with
@Deathcrow as even if someone can ABX at the higher bit rates (say about 160kbps+ on Opus) it's almost surely a minuscule difference on a tiny part of the track and you got to focus quite hard to notice it etc which basically defeats the purpose of lossy audio which is basically to get transparent or near transparent sound at the lowest possible bit rates. plus, how many people have more than average-ish range equipment? which further suggests using those higher bit rates (say beyond 128kbps) are largely a waste of storage space. I know some might say that storage space is cheap, and while that's true, it's more of the thought of it that you might as well make the best use of the storage space you do have instead of wasting it simply because you got storage space to burn.
that's basically why when it comes to Opus I stick to this general guideline because I feel one can't really go much outside of what I suggest below because either sound quality will start to be a bit more suspect (like going below 64kbps) or efficiency will start to drop off quite a bit (going above 128kbps)...
-Minimum = 64kbps (this seems to be my suggested minimum given the testing I have done with my limited equipment(Klipsch Pro-Media speakers/Sony MDR-NC7 headphones). but with that said... if one really wants to push it, maybe 48kbps, but I feel 64kbps is safer over a wider range of music for a minimum bit rate and unless someone is really tight on storage space it's not worth risking the sound quality hit when 64kbps is already quite efficient with storage space and should be noticeably safer over a wider range of music)
-Default = 96kbps (basically the general sweet spot of the Opus encoder for a wide range of music and at this bit rate it's high enough to where the overall sound is quite strong. hell, the Opus Wiki page even says 'approaching transparency' which says a lot)
-Maximum = 128kbps (unless one is super picky about sound quality going beyond this is largely a waste of storage space in my opinion. I suspect the users who use beyond 128kbps are probably that type who obsess over tiny details as even the Opus Wiki page says 'very close to transparency')
NOTE: I generally use 96kbps and occasionally use 64kbps. I just included 128kbps as a bit of a buffer for those a little more concerned with sound quality than storage space but still want some level of efficiency as if someone is not satisfied with 128kbps for lossy audio with Opus they are probably better off going lossless if possible.
I would not claim to be a expert at ABXing either but I can tell given my setup (Klipsch Pro-Media speakers (above average speakers) on my computer and for portable listening Sony MDR-NC7 headphones (average headphones) on a older smart phone) you can't really go wrong with Opus @ 64kbps/96kbps/128kbps overall as while I can notice some slight differences in sound on my Klipsch Pro-Media speakers @ 64kbps here and there it's minimal and when just straight up listening to the song this setting is quite usable especially if one is a bit more concerned with maximum efficiency but don't want to risk sound quality too much either. or look at it another way... I am confident that if you had a random person over to your house listening to Opus music encoded at 64kbps on a decent set of speakers (or okay-ish set of headphones) that they would not notice any obvious flaws in the overall sound as the overall core sound of the song is pretty stable and it's usually certain things you can occasionally notice but even then I would say that the bulk of the stuff one can notice, you need to be hearing the lossless file back and forth between the 64kbps Opus file vs the FLAC file as when casually listening it's easily good enough although I generally go with 96kbps as a safety margin as at 96kbps that should be just about guaranteed to please many people on a wider range of equipment etc. like just seeing random peoples comments on these forums it seems many would be pleased with 96kbps as I think even those who are more picky about sound quality would still find 96kbps to be a respectable minimum.
This kind of "spot the difference" game isn't that interesting to me anyway, particularly when it all comes down to listening to the same 2 seconds of a track 50 or more times until you can make out a tiny difference. At that point it's already way, way, beyond what I'd want out of a lossless codec in terms of quality and I'm more than satisified. I'm not going to care about that when I'm just listening to the music.
Exactly!
basically it's pretty much about finding the smallest file size without any obvious hit to the sound quality when your just sitting back and enjoying your music straight up and in this regard I think 64kbps or 96kbps settings are pretty much where it's at with Opus with nothing more than 128kbps at the most as efficiency is mostly shot beyond 128kbps with Opus and defeats the purpose of lossy audio. like years ago, when MP3 was more used, using bit rates higher than 128kbps was understandable but not with modern encoders like Opus/Apple AAC. but if one is still using MP3 I suggest LAME v5 (130kbps) as a minimum (maybe as low as v7 (100kbps) if you want to live on the edge) although V4 (165kbps) or V2(190kbps) is probably closer to a sweet spot for MP3.
p.s. with that said... I still appreciate the effort of those who do it (like listen to those sound samples over-and-over) to fine tune these modern encoders like Opus.
For a second I imagined to be able to hear a difference, but ABX trials quickly proved me wrong. IMHO if there's an audible difference it's so minute that the smallest lack in concentration makes it go away.
Yeah, I noticed that here and there while doing ABX stuff is that fatigue sets in and usually at that point you know the sound quality is pretty strong when you can't easily pick up differences in the sound and have to start really focusing to notice differences and at this point you can be confident when just sitting back and enjoying your music that the sound quality will be easily good enough as it's not like your going to be sitting there enjoying it and then suddenly think 'wow, that part sounds bad I need a higher bit rate'.
because not long ago I was testing out some music at 64kbps with Opus v1.3 Release Candidates and when switching back and forth between the lossy and lossless track I could notice some difference but when I don't do that and just listen to the Opus track straight up without comparing to the lossless file, it sounds pretty good overall. like nothing obviously negative stands out to me when listening to the song overall and even if I try to focus a little bit it's still up to a standard I am not going to worry about it since the overall sound quality is still strong as it's not muffled etc. like nothing grabs your attention in a negative way.
Is that something that would be acceptable for a studio recording?
Oh you sweet summer child... lol. Yeah, that's pretty normal for any modern metal mastering. And I say this as a pretty big metal fan myself - I have just resigned myself that it will usually (there are some rare exceptions) be crap in terms of loudness, but don't get me started on a rant.
The only thing "odd" I noticed about the flac is that it is 48khz, which means that it may have been upsampled.
It's a very common offense indeed but exceptions to this "rule" are not so rare, it's mostly the mainstream bands (or their labels) who choose to publish degraded-but-"loud" recordings.
There's even an (unpopular) opinion that by dismissing these degraded records immediately without further examination, one will have more free time left to listen to some original and less "commercial" music.
Also, ABX logs are quite easy to forge, so it's not even certain that that guy actually hears the difference under normal listening conditions.
Is that something that would be acceptable for a studio recording?
Oh you sweet summer child... lol. Yeah, that's pretty normal for any modern metal mastering. And I say this as a pretty big metal fan myself - I have just resigned myself that it will usually (there are some rare exceptions) be crap in terms of loudness, but don't get me started on a rant.
The only thing "odd" I noticed about the flac is that it is 48khz, which means that it may have been upsampled.
It's a very common offense indeed but exceptions to this "rule" are not so rare, it's mostly the mainstream bands (or their labels) who choose to publish degraded-but-"loud" recordings.
There's even an (unpopular) opinion that by dismissing these degraded records immediately without further examination, one will have more free time left to listen to some original and less "commercial" music.
Also, ABX logs are quite easy to forge, so it's not even certain that that guy actually hears the difference under normal listening conditions.
How can you forge an ABX test?
How can you forge an ABX test?
foo_abx 2.0.5 report
foobar2000 v1.4
2018-10-24 23:44:23
File A: silence0.flac
SHA1: e2b0876465efb93bbcea63f0ee8e17c845ec31c4
Gain adjustment: +0.00 dB
File B: silence1.flac
SHA1: e2b0876465efb93bbcea63f0ee8e17c845ec31c4
Gain adjustment: +0.00 dB
Used DSPs:
FreeSurround, Channel Mixer, Matrix Mixer
Output:
DS : Primary Sound Driver
Crossfading: NO
23:44:23 : Test started.
23:44:25 : 01/01
23:44:28 : 02/02
23:44:29 : 03/03
23:44:30 : 04/04
23:44:30 : 05/05
23:44:31 : 06/06
23:44:32 : 07/07
23:44:33 : 07/08
23:44:35 : 07/09
23:44:36 : 07/10
23:44:37 : 08/11
23:44:38 : 09/12
23:44:39 : 10/13
23:44:41 : 10/14
23:44:43 : 11/15
23:44:45 : 12/16
23:44:45 : Test finished.
----------
Total: 12/16
Probability that you were guessing: 3.8%
-- signature --
515593c70547bd8627e11110e18d13439240199f
====
Took me less than 5 minutes. Note that the files are identical (there's probably an even better way than stupid brute force).
Hi i'm the guy from the reddit post, I didn't create and account and post before because my english is bad, sorry about that.
The only thing "odd" I noticed about the flac is that it is 48khz, which means that it may have been upsampled.
I read one time, I think it was in this forum, that in a abx test you should use tracks with the same frecuency (also for the spectrogram comparison) so I resample it.
Yeah, but later on he went further to say he was able to discern it at 320kbps.
Only with mp3 at 320k for some tracks, I can't tell the difference between aac at 320k and flac
if someone can ABX at the higher bit rates (say about 160kbps+ on Opus) it's almost surely a minuscule difference on a tiny part of the track and you got to focus quite hard to notice it
For me below 140k opus sounds mechanized (I don't know how to describe it), Above 160k still some instruments are hard to notice, until ~224k it sounds good but different from the original track (I want to listen to the original track, not a similar one) especially all the cymbals.
Also, ABX logs are quite easy to forge, so it's not even certain that that guy actually hears the difference under normal listening conditions.
Sorry if you think that, I don't know another way to prove it, here I recorded my desktop while doing it, I did it as fast as I could so you not get bored (missclicked once)
https://1drv.ms/v/s!AmL_n_RdQAWKhoZjPEtjJ5k_6NV5Yg
Yeah, but later on he went further to say he was able to discern it at 320kbps.
Clarified as MP3 above, thanks Opmox. Different from Opus, what you claimed
@Opmox For me below 140k opus sounds mechanized (I don't know how to describe it), Above 160k still some instruments are hard to notice, until ~224k it sounds good but different from the original track (I want to listen to the original track, not a similar one) especially all the cymbals.
Yeah, it appears some can still detect differences in about 160-192kbps range but seems to be quite minimal given the Opus Wiki page says the following...
96kbps = Approaching transparency
128kbps = Very close to transparency
160kbps to 192kbps = Transparent with very low chance of artifacts (a few killer samples still detectable).
given that info, which I would assume is pretty accurate, I would imagine the vast majority of people would be inline with those standards as that seems to take into account those more sensitive to sound quality. so while your free to use whatever bit rate you want to, maybe your a exception to the norm especially given you said you need 224kbps before things are pretty much fully transparent?
also, correct me if I am wrong, but it seems like even for you once you reach 160kbps things are quite strong for the overall sound quality and you said cymbals are not perfect for you til 224kbps, correct? ; but the thing is, I would imagine there must be a certain bit rate that once you reach it, it's at a point that when you just listen to the song without comparing it to the lossless file that they are not something that would bother you, right? ; because I think if it still bothers you, it's probably more of the thought that you know they are not perfect for you (given you know what your ABXing told you) than the actual drop in sound quality bothering you straight up.
thanks for your time.
p.s. but don't worry if you can't describe everything as I am sure I would be similar as while I can describe a little here and there I could not describe every difference I can hear with the exact technical term etc as some stuff people post around here is simply beyond me ;)
I want to listen to the original track, not a similar one
That's fine and all but since lossy audio is typically used on-the-go, one won't be in optimal listening conditions etc. so any small differences in sound quality likely ain't going to be a problem for most people especially once someone has to really start focusing on the music to notice small differences and at that point one is not really enjoying the music. because it's not like one would notice plenty of flaws in the overall sound once you reach a certain bit rate as it's mostly about cleaning that stuff up and trying to use the least amount of bit rate to do it.
or look at it this way... if you encode some random music tracks with Opus @ 128kbps or 160kbps, without doing a ABX type of test (like when you swap back and forth between the lossless and lossy file), can you clearly notice any problems with the overall sound? ; if not, it just seems like largely a waste of storage space to clean up a tiny bit of sound difference for quite a bit more bit rate. like even if some cymbal stuff was a little off, it's not like someone would focus too much on those while a song is playing when your just sitting back enjoying your music as at that point your more focused on the overall sound/singers voice etc. or another thing, say it's cymbals that tend to be the main issue for you... I would imagine once you reach a certain bit rate those will sound close enough to the lossless track at which point it would be easily good enough considering when your listening to the music you will be more concerned with the overall sound/singers voice, like I was saying.
so while it might vary a bit from person-to-person, there must be a certain bit rate you reach where sound quality gains diminish quite a bit and bit rate starts to spike for minimal sound quality improvement(?). for me, and a fair amount of others around here, we seem to reach this point at 96kbps. which is basically the sweet spot for a fair amount around here. but I can get by with 64kbps if I had to though but it's starting to gamble a little (64kbps is mostly a good option for me when I want something to use minimal storage space but without sacrificing quality much from my usual 96kbps choice and it's 1/3rd less storage space with 64kbps instead of 96kbps). or put it this way... given the polls etc I have seen over the year it seems the vast majority of users around here seem to like Opus @ 96kbps or 128kbps with some occasionally going higher or lower than those two settings as I basically formed my own suggestions based partially on my own opinion and the general info around here with my 96kbps as default, 64kbps min, 128kbps max for general Opus encoding and I tend to keep efficiency in mind to.
just some thoughts and have a good day ;)
Also, ABX logs are quite easy to forge, so it's not even certain that that guy actually hears the difference under normal listening conditions.
Sorry if you think that, I don't know another way to prove it, here I recorded my desktop while doing it, I did it as fast as I could so you not get bored (missclicked once)
https://1drv.ms/v/s!AmL_n_RdQAWKhoZjPEtjJ5k_6NV5Yg
Please pay attention when quoting, I didn't write this. That comment was made by "magicgoose".
Sorry if you think that, I don't know another way to prove it, here I recorded my desktop while doing it, I did it as fast as I could so you not get bored (missclicked once)
Thanks for doing a video.
A 10/16 score is really bad (sounds like guessing to me). Maybe you really missclicked once, but it's typical in this scenario to come up with excuses (brain is trying to resolve your cognitive dissonance).
Think of it like this: Even under ideal conditions you mistook the 'bad' encode as the original 5-6 times, which means the difference - even if you can discern it sometimes - must be minuscule. A car driving past your window while listening is probably a thousand times worse than any artifact introduced by opus.
Did you do the ABX test for the reddit post multiple times before you got 12/16?
For me below 140k opus sounds mechanized
You used a quite harsh word (mechanized) here. If this is true, you should be able to easily accomplish a 20/20 ABX score in a 140kbps opus vs FLAC trial. Can you do that? I'm genuinely interested, because when I use a descriptor like that I will be able to spot the difference a 100% of the time and not just occasionally.
@Deathcrow Hell, that's a good point in your reply to his 140kbps 'mechanized' claim with Opus as that does seem to suggest he can easily notice the difference on any random kind of music and not just a small amount of songs/parts of a song.
p.s. on my best guesstimate for what he means by 'mechanized', I would imagine I would be more in the 32kbps range or so (maybe 48kbps tops) with Opus before it hits that point where I can more easily notice the sound quality hit etc.
Please pay attention when quoting, I didn't write this. That comment was made by "magicgoose".
My bad sorry, it was my first post here.
Thanks for doing a video.
Did you do the ABX test for the reddit post multiple times before you got 12/16?
No, first try, same with the video, but the post on reddit was with the file encoded by anita22 which is ~212k and I did it in 7 min, in the video I used the 256k file in 5 min.
You used a quite harsh word (mechanized) here. If this is true, you should be able to easily accomplish a 20/20 ABX score in a 140kbps opus vs FLAC trial. Can you do that? I'm genuinely interested, because when I use a descriptor like that I will be able to spot the difference a 100% of the time and not just occasionally.
I generalized there, I said below 140k, which means it is noticeable let say at ~96k and almost not noticeable at that bitrate.
Yeah, it appears some can still detect differences in about 160-192kbps range but seems to be quite minimal given the Opus Wiki page says the following...
96kbps = Approaching transparency
128kbps = Very close to transparency
160kbps to 192kbps = Transparent with very low chance of artifacts (a few killer samples still detectable).
Yeah, that's very close to what I said.
Still the initial issue was that I got a 3.8% probability of guessing with a 212k file (tried again next day, same result), while with aac same bitrate I got 59%, it was very hard to notice any difference and I knew I was guessing.
Any lossy encoder has its own difficult to encode samples.
Make a claim aftet testing 20 samples. 1 or 2 samples are nothing.
Please pay attention when quoting, I didn't write this. That comment was made by "magicgoose".
My bad sorry, it was my first post here.
Thanks for doing a video.
Did you do the ABX test for the reddit post multiple times before you got 12/16?
No, first try, same with the video, but the post on reddit was with the file encoded by anita22 which is ~212k and I did it in 7 min, in the video I used the 256k file in 5 min.
You used a quite harsh word (mechanized) here. If this is true, you should be able to easily accomplish a 20/20 ABX score in a 140kbps opus vs FLAC trial. Can you do that? I'm genuinely interested, because when I use a descriptor like that I will be able to spot the difference a 100% of the time and not just occasionally.
I generalized there, I said below 140k, which means it is noticeable let say at ~96k and almost not noticeable at that bitrate.
Yeah, it appears some can still detect differences in about 160-192kbps range but seems to be quite minimal given the Opus Wiki page says the following...
96kbps = Approaching transparency
128kbps = Very close to transparency
160kbps to 192kbps = Transparent with very low chance of artifacts (a few killer samples still detectable).
Yeah, that's very close to what I said.
Still the initial issue was that I got a 3.8% probability of guessing with a 212k file (tried again next day, same result), while with aac same bitrate I got 59%, it was very hard to notice any difference and I knew I was guessing.
Sorry to have dragged you here! I would like to point out you have said AAC was better than Opus for you. Could someone explain the encoding differneces that could explain this? Do you really prefer AAC to Opus?
You should probably try ABX-ing with negative gain to prevent clipping in the lossy file. As the source is extremely broken in terms of DR and RMS, you might just hear differences because after encoding the waveform goes above 0 dBFS all the time. Foobar2000 makes it kind of easy to do the correct way. (scan&tag all files for ReplayGain then check the ReplayGain checkbox in ABX dialog)
This specific .opus file from your link has 1.169 (+1.36 dB) peak which could cause audible problems if played as it without any negative gain.
AAC file has similar problem, though.
As both codecs make no attempts to constrain the waveform into -1..1 bounds, judging them by this ability would be incorrect and useless, so one should prevent this clipping when comparing to avoid picking up artifacts irrelevant to actual lossy coding.
The same is true for all current (and more or less widely known) lossy codecs, with exception of very simple codecs which do not do any transform nor band splitting, such as WavPack or different kinds of ADPCM...
Say he can still ABX after a level adjustment, which wouldn't surprise me in the least, what then? It's clear you're incredulous, but let's not start grasping at straws.
I'm with IgorC: there needs to be an increased sample size for this to have meaning. But what if it does? Will anything change or will we simply accept that some people are more gifted than others? Maybe both?
Sorry to have dragged you here! I would like to point out you have said AAC was better than Opus for you.
For a bitrate less than ~140k opus sounds a little better than AAC, but after that I would choose AAC (FDK).
Ok I searched for a better track, with less clipping, I found out that all of the track that I wanted to test had clipping, I found one that doesn't have it (I think), it was very difficult tho.
This is the result, flac vs opus 256k
foo_abx 2.0.5 report
foobar2000 v1.4
2018-10-26 14:12:52
File A: 03 - The Haunting.flac
SHA1: 915134ef0228944bb2c4f3bda89f2779aa7c6ef9
File B: 03 - The Haunting.opus
SHA1: bec3897e2ed175432f6d7d0f9ff916960c2668c5
Output:
DS : Primary Sound Driver
Crossfading: NO
14:12:52 : Test started.
14:14:00 : 01/01
14:14:19 : 01/02
14:14:43 : 02/03
14:15:15 : 03/04
14:16:20 : 04/05
14:16:53 : 05/06
14:17:44 : 06/07
14:17:58 : 07/08
14:19:24 : 08/09
14:19:40 : 08/10
14:20:01 : 08/11
14:20:35 : 09/12
14:21:14 : 09/13
14:21:33 : 09/14
14:22:00 : 10/15
14:22:50 : 11/16
14:22:50 : Test finished.
----------
Total: 11/16
Probability that you were guessing: 10.5%
-- signature --
d7eb4a5197f0a1deb45593faaecdc81c2e6db98a
The files (https://1drv.ms/f/s!AmL_n_RdQAWKhoZkWTtxMfVk-Nh09g).
I'm gonna test with aac now...
I respect your dedication and honesty in further investigating this. I assume here again that this ABX trial was your first and only attempt? How do you yourself interpret this 11/16 result in terms of you ability to discern the lossy file from the source?
Good luck with the AAC trial... What I would like to see is you attempting the same file at Opus with ~128 kbit/s. I'm curious at what bitrate your ABX results become more definitive.
I respect your dedication and honesty in further investigating this. I assume here again that this ABX trial was your first and only attempt? How do you yourself interpret this 11/16 result in terms of you ability to discern the lossy file from the source?
Second, the first actually gave me a better result (didn't save it sorry) I didn't recorded the 1st because I thought that with that song I was not going to achieve a good result, I thought the same while testing, like a ~60% guessing, the result impress me.
foo_abx 2.0.5 report
foobar2000 v1.4
2018-10-26 15:22:52
File A: 03 - The Haunting.flac
SHA1: 915134ef0228944bb2c4f3bda89f2779aa7c6ef9
File B: 03 - The Haunting.m4a
SHA1: e04cb79dfa5207803234e427ea6b545cbf3f326d
Output:
DS : Primary Sound Driver
Crossfading: NO
15:22:52 : Test started.
15:25:44 : 01/01
15:27:37 : 01/02
15:27:55 : 02/03
15:28:35 : 02/04
15:28:50 : 03/05
15:29:06 : 04/06
15:29:30 : 05/07
15:30:37 : 06/08
15:30:56 : 06/09
15:31:55 : 06/10
15:32:26 : 07/11
15:33:19 : 08/12
15:33:41 : 09/13
15:34:17 : 10/14
15:34:56 : 10/15
15:35:15 : 10/16
15:35:15 : Test finished.
----------
Total: 10/16
Probability that you were guessing: 22.7%
-- signature --
f9c0738d0162af46e68a69650377e0168b0039b1
:o This test felt like a 100% guessing, I don't know what to think
:o This test felt like a 100% guessing, I don't know what to think
Maybe that this is a very useful experience for you where you're learning a lot about (your) perceptual biases and the placebo effect?
There's good reason why this community strongly encourages the use of objective tests like ABX (and other's don't). Props to you for standing up to the challenge.
What I would like to see is you attempting the same file at Opus with ~128 kbit/s. I'm curious at what bitrate your ABX results become more definitive.
foo_abx 2.0.5 report
foobar2000 v1.4
2018-10-26 16:05:33
File A: 03 - The Haunting.flac
SHA1: 915134ef0228944bb2c4f3bda89f2779aa7c6ef9
File B: 03 - The Haunting.opus
SHA1: 9f600815e8a92b59f9010e66cc29115c70fd613c
Output:
DS : Primary Sound Driver
Crossfading: NO
16:05:33 : Test started.
16:07:40 : 01/01
16:07:45 : 02/02
16:07:54 : 03/03
16:07:59 : 04/04
16:08:08 : 05/05
16:08:15 : 06/06
16:08:23 : 07/07
16:08:29 : 08/08
16:08:37 : 09/09
16:08:46 : 10/10
16:08:54 : 11/11
16:09:00 : 12/12
16:09:06 : 13/13
16:09:18 : 14/14
16:09:28 : 15/15
16:09:37 : 16/16
16:09:37 : Test finished.
----------
Total: 16/16
Probability that you were guessing: 0.0%
-- signature --
d0abcffb68d31b08201c6b0537d53f979b8a85a5
This was was easy because I found an artifact and then only listen to the track X :D
Yep, that's certainly more definitive, lol.
I'm quite impressed, I don't think I can do that unless someone points out a certain thing that I have to listen for. There's even a slight indication that you can sometimes discern lossy from lossless at much higher bitrates. Not too shabby overall.
Yeah, I simply got no words. So... what are the implications of this? Is Opus technically worse than AAC at higher bitrates? Is it just that he's gifted? Is it that AAC's artifacts are less difficult to detect?
I guess we should stick with what works better for each of us, and for me Opus @ 96kbps is absolutely perfect, but this really has left me with many doubts.
@Opmox This test felt like a 100% guessing, I don't know what to think
Yeah, that's why when I do an occasional ABX test I generally don't proceed to the next guess unless I am pretty confident I am hearing a difference as that helps get you in the ball park of when your straining and starting to guess vs know your hearing a difference etc.
but after seeing your other post it seems you can do Opus @ 128kbps since you got everything correct with 16 tries to. but I am curious... can you do that (like ABX Opus @ 128kbps) on any random song, or just certain songs? ; but lets just say you can consistently ABX Opus @ 128kbps... I am wondering at which point you start to strain and are more or less guessing or semi-guessing? ; like say 160kbps or 192kbps?
but one thing I noticed, at least for myself, is usually when your pushing the limits of your hearing or thereabouts, it almost seems like fatigue sets in when you start really focusing for any length of time and stuff you might have been able to detect initially, you start questioning whether your hearing a difference or not etc.
p.s. for me, given my general testing here and there on my speakers/headphones (mainly my Klipsch Pro-Media speakers on the PC (which I would say are above average speakers for sure))... I would say once I hit 96kbps I doubt I can notice the difference (at least not reliably for sure) as I think I can hear a difference at 80kbps here and there but even here is not easy for me. hell, I still consider 64kbps pretty good overall as while it's worse than 96kbps, as 96kbps is safe for me, I would say 64kbps ain't that much worse and is 1/3rd less bit rate. like when just listening to a song straight up, the overall core sound is intact at 64kbps on my Klipsch Pro-Media speakers as I use 64kbps here and there but generally stick with 96kbps to be safe.
NOTE: I know I should generally use headphones but I would guesstimate I won't be much either way given I would say my headphones (Sony MDR-NC7) are average-ish and I would not expect my preferred 96kbps Opus bit rate change much, if any, especially given the general Opus Wiki page says 'Very close to transparency' for 128kbps and I assume these kinds of claims are probably with people who are above average hearing (or no worse than the typical person) and the like which probably covers easily most people and I doubt my hearing is anything special. like decent(as I doubt my hearing is bad for my age bracket(I am 39 years old)), but nothing special. plus, I imagine some people who do testing for people to improve Opus etc probably have better than average headphones to.
Yeah, please, try another song and let us know.
Moderation: link removed.
<= 30s, per TOS#9, or do I have to remove the link?
I have found another flac for the song. This time, there's no upsampling.
Here it is the link (https://drive.google.com/open?id=1REriVurGydnQlKP1M3z3txfvTKK9BLiY).
<= 30s, per TOS#9, or do I have to remove the link?
I was afraid of that, files removed.
I have found another flac for the song. This time, there's no upsampling.
I don't know, looks the same to me https://i.imgur.com/Y9Mpkub.png. Edit. misread, you mean 44100hz? I thought that the problem was the clipping
can you do that (like ABX Opus @ 128kbps) on any random song, or just certain songs? ; but lets just say you can consistently ABX Opus @ 128kbps... I am wondering at which point you start to strain and are more or less guessing or semi-guessing? ; like say 160kbps or 192kbps?
No, all depends of the song, e.g. I tried Toto - Africa and I got 11/16 at 128k
but one thing I noticed, at least for myself, is usually when your pushing the limits of your hearing or thereabouts, it almost seems like fatigue sets in when you start really focusing for any length of time and stuff you might have been able to detect initially, you start questioning whether your hearing a difference or not etc.
Yeah sometimes the instrument sound you're listening to remains in your head, I usually keep playing the song and focus on another instrument and section. Also sometimes, like yesterday I started listening artifact in all the songs, even in the flac, when that happens I cannot continue.
One question...
Opus has never prioritized complete transparency at higher bitrates. We have AAC and MP3 that do a great job. Opus sounds very good as well.
Like all codecs, I'm sure they will wring out as much performance as possible over time. But high bitrate is the land of diminishing returns and there is so much room for improvement/much to be gained at lower bitrates.
Is this true? :(
Opus has never prioritized complete transparency at higher bitrates. We have AAC and MP3 that do a great job. Opus sounds very good as well.
Like all codecs, I'm sure they will wring out as much performance as possible over time. But high bitrate is the land of diminishing returns and there is so much room for improvement/much to be gained at lower bitrates.
Is this true? :(
[/quote]
Didn't know about that. Bear in mind Opus is extremely new when compared to AAC. Maybe they improve at higher bitrates. I don't know. It sounds transparent to me at 96kbps.
It's easy to fall into wrong conclusion by testing just one or two samples
Results for one single "Girl" sample
https://listening-test.coresv.net/s/tr05.png
Global results for all 40 samples
https://listening-test.coresv.net/s/scores_by_tracks_en.png
Have a look for yourself https://listening-test.coresv.net/results.htm
@Opmox Thanks for the info. but I just read on another part of the forums (posted today on the 'Opus 1.3 is out!' thread) which I think speaks volumes...
"There's a good reason we haven't seen 128 kb/s AAC/Opus listening tests. It's because everything's so close to transparent you'd need hundreds of samples and hundreds of listeners to get anything statistically significant (as opposed to everything being transparent except for one codec randomly hitting a killer sample). Even the last 96 kb/s test was pushing it." - jmvalin (Opus developer)
a reply to that quote from user IgorC...
"It’s true. I can explain in which exactly way the last two 96 kbps tests were pushing it.
Most of average listeners hear artifacts at 48-64 kbps for HE-AAC and Opus encoders.
The listeners who have participated in 64 kbps have commented that they couldn’t submit a high number of results if any for 96k tests (because they hardly could ABX it).
During 96k tests a number of submitted results per each week drop to less than 50% than during 64 kbps test!
48-64 kbps tests- average listeners prevail.
96 kbps tests - average listeners + experienced listeners (roughly ~50%/50%)
128 kbps test - highly experienced and trained listeners mainly (hence not representative as a real-life scenario)"
I think that's pretty good info when it comes to the general public at large and pretty much sums me up to as 64kbps (based largely on my Klipsch Pro-Media speakers) is about the max I would probably do okay-ish at ABX tests. but once I hit 80kbps (assuming we stick to 16kbps each change of bit rate standard(i.e. 48kbps/64kbps/80kbps/96kbps etc)) it's really pushing it (I would expect to largely fail ABXing on random songs on my speakers at 80kbps, outside of certain parts in some songs etc) and 96kbps seems to be transparent for me, and given the above info in the quotes, it seems many people find it difficult to ABX at 96kbps and once a person hits 128kbps your already comfortably out of the range of most people (if not vast majority?).
so that info is pretty much inline with my signature on Opus bit rates... 64kbps minimum, 96kbps (default), 128kbps maximum. I would say that's a pretty good default system for most people because your not going to far either way from that general center point (i.e. sweet spot) of 96kbps. so with those three choices your always having a decent balance of sound quality/storage space combo without things going too far either way. like with 64kbps your leaning towards storage space with sacrificing sound quality a little and 128kbps your sacrificing storage space a little for sound quality a bit but your not going too far either way of that general point (i.e. 96kbps) many (if not most? ; if not vast majority?) people start to really blur the line on detecting differences between Opus and lossless source on music. so in other words... most people should probably avoid bit rates lower than 64kbps or higher than 128kbps for music as a general guideline.
p.s. but like I have said before... 64kbps is solid enough overall especially when you don't compare it to the lossless source in ABXing back and fourth between samples (like when you click A and B) and just sit back and enjoy. like 64kbps is noticeably safer on sound quality than 48kbps for a minimum increase to file size. I know one could say there is a 25% storage space savings between 48kbps vs 64kbps, which sounds like a quite a bit of storage space savings, but when your already at pretty low bit rates like that, it's not that much storage space savings in the real world (although going from 48kbps to 64kbps does give you a decent bump in sound quality over a wider range of music) unless you got a boatload of music as I would say choosing something like 64kbps over 96kbps might be worth considering though if your tight on space or don't mind sacrificing sound quality a little bit for the extra storage space savings. but going from 64kbps to 32kbps is not worth it as while it halves the storage space needed, the sound quality hit is a bit too much to where unless one is heavily favoring storage space over sound quality they would not choose 32kbps over 64kbps for music in general unless they had to if they wanted to get plenty of music onto a really limited amount of storage space etc.
Even with very neutral, accurate gear and an experienced ear, would one be able to consistently discern at 160-192?
And at VBR wouldn't 192 have the potential to reach 224?
And at VBR wouldn't 192 have the potential to reach 224?
VBR, if done 100% properly, doesn't relate to any particular bitrate and instead uses whatever amount of bits to reach some specific quality level. Opus authors chose to rank quality in "average bitrate if used on some collection of average music", but that doesn't change the general idea of VBR.