Skip to main content

Topic: Thinking of switching codec - need some advice please (Read 2693 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Lee James
  • [*][*][*]
Thinking of switching codec - need some advice please
Hi guys :) I could do with a little advice, please…

For the last 15 years I've happily chosen to encode my music with LAME mp3 (V0). I've always considered this perfect for me for a few reasons…

My criteria
  • I am fussy about quality - but not to extremes
  • I'm only a casual listener, with cheap earphones, I'm not an "audiophile" when it comes to listening to music
  • I enjoy music, but ultimately my collection is not terribly important to me
  • I'm happy as long as the music sounds like original most of the time - I don't mind the occasional 'imperfection' or 'glitch'
  • I'm not fussy enough to care about the whole "subconscious perception" issue
  • I do have a very large music collection, and even though hard drive space is cheap, I don't like the idea of how much space lossless would take up
  • Another key factor for me is that I'm a "minimalist" and find it satisfying to know that my music files take up as little space as possible, with all the unneeded data stripped away

So LAME mp3 has always been fine for me, on this basis.

But lately I've been wondering if it might be time to switch codec? Maybe there's something much better than mp3 these days? Perhaps ogg or something? I really known nothing about the other lossy formats.

But in particular, something else has been bugging me about LAME mp3… As a musician myself, I have a real interest in how all the parts in the track are mixed. So, while normal listeners will just focus on the lead vocal and main melody - my ears are more sensitive to "every single part of the mix" - I will often be paying attention to individual instruments in the background, or individual notes in chords - things that are perhaps really subtle in the mix - two instruments blended together, or one very quiet instrument in an orchestra hard-panned to the far right, just about audible in one ear. These are details that most people wouldn't care about. And then sometimes I will listen just to an isolated channel (left or right) just to hear what is going on in each channel.

So I guess I'm asking: Is LAME V0 good enough for me? I'm not obsessively fussy about the music being "perfect", but I do want to be able to hear all the parts and all the harmonies, in all the mix, across both the channels - including all the subtle nuances of every instrument (something a regular listener may not care about).

And can LAME V0 be trusted for playing just the left or right channel in isolation? Or does it start to break down when you do that? (To my ears, LAME V0 does sound identical to the original WAVs, but I haven't done enough testing to put any confidence in my own conclusions.)
  • Last Edit: 27 May, 2017, 02:06:26 PM by Lee James

  • bennetng
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #1
Do you make music transcription? Some audio processing plugins manipulate the phase of stereo signals to emphasize or minimize some instruments panned on a specific position to make transcription easier, the most common one is the so-called "vocal removal" technique which assumes vocal is mono and centered on the stereo field.

https://youtu.be/WvqcrAWflJk

With mp3 or some other lossy codecs such techniques could yield poorer performance.

Sometimes you may also want to slow the song down to pick up some fast notes, lossy codecs could also yield poorer performance.

However, in my experience, if a stereo mp3 is transparent (audibly lossless), it will still transparent when only one channel is being played.

  • Klimis
  • [*]
Re: Thinking of switching codec - need some advice please
Reply #2
Silly comment but if
Do you make music transcription? Some audio processing plugins manipulate the phase of stereo signals to emphasize or minimize some instruments panned on a specific position to make transcription easier, the most common one is the so-called "vocal removal" technique which assumes vocal is mono and centered on the stereo field.

https://youtu.be/WvqcrAWflJk

With mp3 or some other lossy codecs such techniques could yield poorer performance.

For only that issue I have found that AAC-LC is the best option by far from any other, but I repeat, only for that issue.

If your don't have any other of the common codec-switch issues like running out of storage on that particular device, need a more efficient codec for given (lower than transparent) bitrate or compatibility, then I don't see why you would need to re-rip everything from your catalog if everything is already transparent to you, you'll gain nothing.

Re: Thinking of switching codec - need some advice please
Reply #3
The best advice I can give is: Don’t panic! ;)

As well tested as LAME is it’s extremely unlikely that V0 is anything but perfectly transparent – maybe you’ll find problematic samples, but they’ll be few and far between.

But there’s nothing wrong with doing some ABX tests yourself to get back your peace of mind. If you’re not able to hear a difference between the original and the encoding in a double-blind test done in your usual listening environment(s) with the music you listen to all the time, then there isn’t any point in switching formats, is there?

If even that’s not enough to combat the paranoia, then consider solving the problem once and for all by switching to a lossless format (e.g. FLAC).

  • halb27
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #4
a) I'm not sure but what your post tells me is that you do worry about quality and - if it were possible - you would prefer to go lossless.

b) Lame 3.99.5 -V0 is expected to be absolutely fine for nearly everything. You can look at the results of the last public test @128 kbps to see that Lame3.99.5 is very good even at this moderate bitrate.
If you like to go a little step further with Lame for some very special samples you can try my Lame variant lame3995o and use -Q1 (224 kbps on average for pop music) or -Q0 (317 kbps) or anything in between, for instance -Q0.5.

c) If you would like to use a more modern codec you can use opus (or AAC). opus --bitrate 128 is expected to be transparent for nearly every kind of music. You can give some headroom like using --bitrate 192 or even more to make sure that even tracks extremely difficult to encode are handled fine. According to a listening test I performed recently opus has the great merit that some tonal problems I care about are handled fine at a ~100 kbps (avg.)
 setting where other codecs need a significantly higher bitrate. (I did care about these when developing lame3995o, and it takes a ~170 kbps (avg.) setting to handle them fine).

d) If you can allow for ~400 kbps lossyWAV+FLAC is a near-lossless option. Same goes for wavPack lossy resp. hybrid. You can go even lower with bitrate when using lossyWAV+FLAC, but in the ~300 kbps region I'd prefer opus (or lame3995o or QAAC).
  • Last Edit: 27 May, 2017, 03:42:02 PM by halb27
lame3995o -Q1.7
opus --bitrate 140

  • Lee James
  • [*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #5
Update:
Sorry guys, I really should have done this before posting the thread, but…

I have just done some blind testing (for the first time in many years) and I am horrified to discover that my ears can actually hear the difference between LAME V0 and WAV! To me, the mp3s sound like they're missing high frequencies, and thus they sound slightly duller (and narrower). In contrast, the WAV sounds less like a recording and more like "real life".

It's only very subtle, but it's there. I did quite a lot of blind testing - sometimes both samples sounded the same, however in the majority of cases I could identify the lossy version. (There were no cases where I mistook the mp3 for the lossy version, I either got it right or I wasn't sure.)

So now, after this, I don't think I can use mp3 again! I'm not about to re-rip everything (some of the CDs I don't even have any more), but I definitely want to up the quality in future.

So my new question is: is there any codec somewhere between LAME quality and lossless? I don't like the idea of my music containing "useless" data, but for me at least, mp3 is not high quality enough. (Funny, I never thought I would say that - I always completely trusted the fact that LAME V0 is regarded as "transparent" - but for me personally, my ears seem tell a different story.)

Do you make music transcription?
Thanks for your answer, which was very helpful and very relevant. Yes, I do transcribe music sometimes, and when I do, I sometimes slow down tracks and use "center channel removal". But this is fairly rare - not enough for it to be important. I fully expect an mp3 to sound poor quality when you mess with it like this. It's only really "normal listening" I'm interested in (or "normal listening" with one channel).

  • halb27
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #6
Do you mind sharing a 30sec. snippet of that track with us? An ABX protocol is also welcome. You must understand that to most people here it sounds strange when someone claims that he can ABX a track @V0 which is not known in advance to be difficult to encode. Especially as you aren't talking about artefacts but about missing highs (which is the usual reasoning of people judging from spectograms). ABXing successfully an ad hoc track can happen, but more evidence is extremely welcome.
BTW which version of Lame are you using, what are your exact encoding parameters, and what is your age?

As for mp3 can you try lame3995o -Q0.5 (same average bitrate as -V0) or -Q0 (317 kbps on avg.) on your track? You can download it  from here. Please provide an ABX protocol.

For an alternative to mp3  try opus --bitrate 192 (or more) and/or lossyWAV+FLAC at a ~400 kbps (avg.) setting.
  • Last Edit: 27 May, 2017, 04:16:44 PM by halb27
lame3995o -Q1.7
opus --bitrate 140

  • bennetng
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #7
(To my ears, LAME V0 does sound identical to the original WAVs, but I haven't done enough testing to put any confidence in my own conclusions.)

Update:
Sorry guys, I really should have done this before posting the thread, but…

I have just done some blind testing (for the first time in many years) and I am horrified to discover that my ears can actually hear the difference between LAME V0 and WAV! To me, the mp3s sound like they're missing high frequencies, and thus they sound slightly duller (and narrower). In contrast, the WAV sounds less like a recording and more like "real life".

Did you ABX a lot of files within two hours and suddenly noticed the nontransparency? ;)

I am more interested to hear some of your transcriptions. It is not totally off-topic. By hearing your transcriptions (not rearrangement or improvisation) and compare with the original piece, it is possible to evaluate your ability to hear individual components of a mix.


  • Lee James
  • [*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #8
Did you ABX a lot of files within two hours and suddenly noticed the nontransparency? ;)
I've been encoding with LAME for 15 years and until today I've always taken it for granted that V0 is "transparent". I've always accepted the wisdom and experience of the body of testers, because they are the real audiophiles and who are going to hear flaws better than I could.

To my ears, LAME always sounded exactly like the original WAV, just based on listening to it, rather than blind testing. I've never done much testing myself, just perhaps a little from time to time, and never had any reason to doubt that V0 is effectively transparent for general listening purposes.

But I cannot deny that my ears have become more trained over the years, in certain ways, simply from working with music so much. Maybe my ability to hear has changed as I got older? I don't know.

Having written this OP today, I figured I would probably get a few people telling me to go away and do some testing myself, so I figured it made sense to do just that. Honestly, I only spent about 20 minutes doing it, not long. But I would say it was enough to convince me that I can tell V0 from WAVs.

I figured that I'm not the only one who can do that. Some people must think they can hear a difference, which is why not everyone likes mp3s. However, if I am very unusual and people doubt my results then I would be happy to do more testing, and people are welcome to send me samples of their own choosing - two lossless files (one of which was originally an mp3), and I will tell you if I can hear a difference :)

BTW which version of Lame are you using, what are your exact encoding parameters, and what is your age?
The LAME version I used was

LAME 3.99.5 (February 28 2012)

Parameters:
-S --noreplaygain -V 0 - %d

Bitrate:
245

Settings:
V0

I'm 40 years old and I wouldn't even say my hearing was great. In my right ear I have very slight tinnitus - a constantly high pitched sound. And my left ear is worse - currently badly bunged up with wax, which limits my high frequencies compared to my right ear! I just saw the doctor about it a couple of weeks ago and she recommended I use oil for 3 weeks then possibly get it syringed (which I don't want to do as it was syringing which caused the tinnitus in my right ear!) Generally I've been ignoring the problem and putting off dealing with it. So my ears are far from a model of perfection! :)

Do you mind sharing a 30sec. snippet of that track with us? An ABX protocol is also welcome.
I used a variety of random tracks from random albums - nothing special about an of them and they all gave the same result, so I don't think it has anything to do with tracks. If anyone could recommend a good file sharing site then I might upload an example of what I listened to?

As for ABX protocols, I don't know what that is so you'll have to explain to me how I can do a proper test? The method I used wasn't particularly scientific:

What I did was I opened both the original WAV and the mp3 in Audacity, renamed the two tracks so that the origin of each track was hidden, jumbled up the two tracks a lot (not always knowing if I had dragged it properly or not). Then I simply played excerpts on loop whilst toggling the "solo" button to switch between the two versions. Certain excerpts sounded the same, but in most cases one would sound clearer, and the one that sounded clearer was always the same track, which turned out to be the original WAV. I did this with a few tracks and the same happened in every case.

Note that, psychologically speaking, I really did NOT want this to happen. I kept wishing and hoping I would get it wrong half the time, because I'd hate to think that there is anything deficient about my mp3 collection. So if I could have somehow subconsciously cheated, this is not the result I would have engineered! But, alas, I cannot deny what happened.

Also note that it is only an extremely subtle difference - nothing major, but there is a difference, according to what I heard anyway. I also apologize if that bothers other people. It bothers me.

You must understand that to most people here it sounds strange when someone claims that he can ABX a track @V0 which is not known in advance to be difficult to encode. Especially as you aren't talking about artefacts but about missing highs (which is the usual reasoning of people judging from spectograms). ABXing successfully an ad hoc track can happen, but more evidence is extremely welcome.
I'm afraid you've lost me a little here. I don't know much about these things, and testing. I don't believe it was an ABX test. And then of course my results could just be coincidence.

For an alternative to mp3  try opus --bitrate 192 (or more) and/or lossyWAV+FLAC at a ~400 kbps (avg.) setting.
Thanks, I will give it a try! :)

I am more interested to hear some of your transcriptions. It is not totally off-topic. By hearing your transcriptions (not rearrangement or improvisation) and compare with the original piece, it is possible to evaluate your ability to hear individual components of a mix.
Thanks for your interest but I don't tend to share my cover versions, I only make them for my own amusement. I sometimes think about sharing them on YouTube, but I'm in no rush to. And they are not supposed to be 100% perfect copies of the original, usually I am seeking to retain the original music but with subtle improvements and embellishments, especially to the production. Then, I also do things like recreating music on the ZX Spectrum, which only has 3 channels - so it sounds very different to the original. (I love the challenge of things like that.)

But I do think my transcriptions are better than most. I usually find other people's cover versions deeply irritating because they almost always get melodies, chords, and timing wrong. MIDI files make me particularly angry, I don't think I've ever heard a MIDI file cover version done properly, they're all appallingly bad! :D (Has anyone ever downloaded a good MIDI file from the internet?)

Though when transcriptions are bad, it's less likely to be caused by the transcriber's hearing, and more likely to be caused by any of these factors…

  • His ability to recognize and remember notes, chords, and timing
  • His knowledge of musical composition and familiarity with instruments
  • His technical ability to reproduce the music
  • The instruments and effects he has access to
  • The amount of time and care he chooses to spend - the more time you spend, the closer it will be
  • Last Edit: 27 May, 2017, 07:40:46 PM by Lee James

  • halb27
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #9
You can send a snippet of a track to h.alb@web.de. I'll give a link here to my web space where I'll put it.

You can do a proper ABX test by using foobar2000. Select the original and the mp3 file in the play list, right click and choose ABX.

As the next step you have to choose how many ABX steps you want to do. Choose for instance 8 for the least painful ABXing. In this case you must be correct with 7 out of these 8 steps in order to successfully ABX an audible difference.

In the ABX dialogue you can always listen to the original as being labeled A and the mp3 version as being labeled B.

Within an ABX step you are given a version X and a version Y, and you have to decide which is the original. You can listen to X, Y, A, B as often as you like within a trial step but finally you have to decide. When finished with your preselected number of ABX  steps you can save the ABX protocol.

Please do a proper ABX procedure like that of foobar. Chance is high you will get at another conclusion.
lame3995o -Q1.7
opus --bitrate 140

  • bennetng
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #10
As for ABX protocols, I don't know what that is so you'll have to explain to me how I can do a proper test? The method I used wasn't particularly scientific:

What I did was I opened both the original WAV and the mp3 in Audacity, renamed the two tracks so that the origin of each track was hidden, jumbled up the two tracks a lot (not always knowing if I had dragged it properly or not). Then I simply played excerpts on loop whilst toggling the "solo" button to switch between the two versions. Certain excerpts sounded the same, but in most cases one would sound clearer, and the one that sounded clearer was always the same track, which turned out to be the original WAV. I did this with a few tracks and the same happened in every case.

I am afraid what you did are not double-blind tests like ABX since you shuffled the playback yourself. Try this one instead:
http://www.foobar2000.org/components/view/foo_abx

The whole thing can be done within this forum like the post below, and this forum allows pretty large attachments.
https://hydrogenaud.io/index.php/topic,112486.0.html

Personally I don't take ABX logs very seriously but since it is a rule of this forum, and judge from the fact you are a long time member since 2003, I suppose you clearly know how this forums works and what a proper ABX test is.

Quote
Thanks for your interest but I don't tend to share my cover versions, I only make them for my own amusement. I sometimes think about sharing them on YouTube, but I'm in no rush to. And they are not supposed to be 100% perfect copies of the original, usually I am seeking to retain the original music but with subtle improvements and embellishments, especially to the production.

So do I. About the transcription/rearrangement thing, feel free to post some when you are comfortable to, for example on the General Music Discussion board. But I'd just starting with share one of my work done in 3-4 hours using only GM2 instruments instead of typing a lot to talk about the limitations of doing such things and post nothing, especially since English is not my native language. Don't worry, I am not going to talk about the instruments/articulations/type of reverb/panning positions etc are different from the original, everyone doing such things have to face some limitations. What I am interested about is how the overall presentation of the transcription is, and how people tackle the limitations. There are no absolute criteria to judge a transcription "good" or "bad", but it is unfair to say yours are superior to others without even posting some of your works, just like doing some proper ABX tests and share the results.

Original:
https://youtu.be/hMnl7VmfWo4

My version:
http://embed.nicovideo.jp/watch/sm10088593

If you have a nicovideo account (free to register), use this link instead since sound quality is better.
http://www.nicovideo.jp/watch/sm10088593

  • halb27
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #11
A small correction to my last post:
When I said A is the original this is only true if it is the upper file of the play list.
So exactly: A is the upper, B is the lower file in the play list among the two files selected.
lame3995o -Q1.7
opus --bitrate 140

  • bennetng
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #12
Also, when doing listening tests related to lossy encodings, be careful with this issue.
https://hydrogenaud.io/index.php/topic,114101.msg939763.html#msg939763

  • halb27
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #13
Unfortunatly this makes things more complicated for the OP.
I guess also with the straightforward procedure he will have a hard time finding something he can ABX.
In case he should really find someting we can have a look at the provided sample and, if the track peak value should give rise to the suspect that the overall playback chain will change volume, we can ask him to manually edit track replaygain values of the original (converted to FLAC) and the mp3 file and ask him to redo the ABXing using replaygain. We can help him to any amount with that in case it should be necessary.
  • Last Edit: 28 May, 2017, 06:53:18 AM by halb27
lame3995o -Q1.7
opus --bitrate 140

  • Lee James
  • [*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #14
Thanks for the replies. I've got a very busy day today so will probably continue with testing tomorrow.

In the meantime, may I ask: are there people who are known to be able to genuinely hear the difference? Or would I be the first - therefore it's much more likely I have tested wrong? (If I have then my apologies, but I don't see how.)

I don't consider my eyesight or hearing to be unusual, but I will say am quite an unusual person, and seem to have more sensitive senses in many ways, including smell, touch, and others. There are some things I can sense that people say is impossible to sense (but I won't go into that here as it would be way off-topic).

  • bennetng
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #15
In the meantime, may I ask: are there people who are known to be able to genuinely hear the difference? Or would I be the first - therefore it's much more likely I have tested wrong?
Rarely but not impossible. Think about anti-virus software, most of them claim something like 95-99% detection rate, but only 1% of virii can mess up the whole computer. There are also false positives as well.

I was able to ABX a 320kbps sample 12 years ago for example.
https://hydrogenaud.io/index.php/topic,112569.msg927661.html#msg927661

LAME V0 could be nontransparent, but if it is, the chance should be pretty small, not something a 40 years old adult will experience frequently.

  • halb27
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #16
You describe to hear a difference on a regular basis for unselected tracks using Lame 3.99.5 -V0. I have never heard before that someone can.
I personally can ABX two specific tracks @V0 (reduced to 1 after having developed lame3995o). I guess everybody can ABX these tracks. But on a regular basis I cannot even ABX V5.
There are people who are extremely sensitive to specific aspects and can ABX these without much pain. I guess I am a bit more than average sensitive to tonal issues, but @V0 everything is fine. This was the main reason for me to develop lame3995o, and with it all my tonal problems are fine with a -Q setting which takes less than 200 kbps on average.

But just let's see what you'll find out doing a proper ABX test.

lame3995o -Q1.7
opus --bitrate 140

  • Lee James
  • [*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #17
OK I've installed the ABX comparator plugin and looking forward to doing some REAL testing tomorrow! :)

I'm excited because either way it's guaranteed to be a great result. If the result shows that I truly can hear the difference, then my discovery will be a valuable contribution to audio science. If, however, it shows that I can't tell the difference, it will mean LAME works and my music collection is properly encoded! (Which is the result I'm really hoping for!)

  • Lee James
  • [*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #18
Hi again guys. Sorry, it's been a couple of weeks now since I said I was going to do an ABX test. I've been really busy but just had time to do one tonight…

I'm posting the log here (below). It says I got 12/16, but someone who was guessing would get it right 8/16 times so 12/16 isn't significant more - or is it? I would appreciate if someone could tell me what these results mean - does it mean I am probably actually hearing a difference, or could it just be guesswork?

The track I tested was Shola Ama - Much Love

foo_abx 2.0.2 report
foobar2000 v1.3.12
2017-06-17 21:50:07

File A: Unknown Artist - 1-02 - Track02.mp3
SHA1: 3c5aaec421e9bbd9b0ca1664fcfb18428337caec
File B: Unknown Artist - 1-02 - Track02.wav
SHA1: 3c350ca31e9bf796647ebb25d01dbfdd6b8d361c

Output:
DS : Primary Sound Driver
Crossfading: NO

21:50:07 : Test started.
21:51:00 : 01/01
21:52:00 : 02/02
21:52:28 : 02/03
21:53:15 : 03/04
21:53:43 : 04/05
21:54:05 : 05/06
21:54:55 : 06/07
21:57:02 : 07/08
21:59:03 : 08/09
21:59:51 : 09/10
22:02:29 : 10/11
22:05:33 : 11/12
22:06:46 : 11/13
22:08:04 : 11/14
22:09:54 : 12/15
22:11:15 : 12/16
22:11:15 : Test finished.

 ----------
Total: 12/16
Probability that you were guessing: 3.8%

 -- signature --
db7e7d19081748f04ed0ab7059df95c0d56bfc4b

  • greynol
  • [*][*][*][*][*]
  • Global Moderator
Re: Thinking of switching codec - need some advice please
Reply #19
Because the probability that you could get the same result or better by flipping a coin is below 1 in 20 (your score, for lack of a better term, was 3.8%) this is considered a passed test.
Is 24-bit/192kHz good enough for your lo-fi vinyl, or do you need 32/384?

  • halb27
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #20
@Lee James:
Do you mind sharing the critical spot with us?
I would like to know whether or not you can hear a difference also with my lame3995o variant using -V0 as well as -Q0.5 (same average bitrate). Can you please try it?
lame3995o -Q1.7
opus --bitrate 140

  • Vivadavid
  • [*]
Re: Thinking of switching codec - need some advice please
Reply #21
Because the probability that you could get the same result or better by flipping a coin is below 1 in 20 (your score, for lack of a better term, was 3.8%) this is considered a passed test.

After reading this thread, I decided to look into ABX tests. However, I'm a bit confused about one particular thing. The Wiki (http://wiki.hydrogenaud.io/index.php?title=ABX) recommends 16 trials, just as Lee James did, and it adds that "a difference is concluded to be heard when 13 correct identifications out of 16 is achieved". Lee James only got 12 out of 16, and despite that, Greynol said the test was passed.
  • Last Edit: 18 June, 2017, 06:54:51 AM by Vivadavid

  • halb27
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #22
In the Wiki Article an error of only 1 percent is allowed whereas the foobar ABX procedure allows for 5 percent.
Being within a guessing probability of 5 percent maximum is what is usually accepted here.
lame3995o -Q1.7
opus --bitrate 140

  • Lee James
  • [*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #23
Because the probability that you could get the same result or better by flipping a coin is below 1 in 20 (your score, for lack of a better term, was 3.8%) this is considered a passed test.
OK, I'll take your word for that. I actually studied statistics and used to know all the probability formulas, but it's been a while and I can't be bothered working it out. :)

I find it somewhat amusing that my hearing isn't what you'd call perfect (I have tinitus in one ear, and the other is severely blocked with wax) and I'm using a really cheap pair of earphones, and yet I was able to do this! :) LOL

I would like to know whether or not you can hear a difference also with my lame3995o variant using -V0 as well as -Q0.5 (same average bitrate). Can you please try it?
Sure, I'd be happy to, when I get a chance… I tried downloading your LAME variant but it said file not found, so if you can let me know where I can get it, that would be helpful thanks. :)

Do you mind sharing the critical spot with us?
If by "critical spot" you mean the part of the song which "gave it away", there was no critical spot - just the whole song.

Here are some notes which may be of interest:

Notes on how I passed the ABX test

  • The song was chosen completely randomly. There was nothing at all about this song which made me think it would be a good test. It just so happened that I was in the middle of converting it to mp3, so I happened to have both the WAV and mp3 files on my computer.
  • While this is the only ABX test I've ever done, I had done some preliminary testing of my own (as mentioned) using Audacity - I listened to about 3 random songs by different artists, and could hear some difference in many portions of all the songs
  • I deliberately did all my testing in the evening because I have a theory (only a theory) that my ears work better at different times of the day - I feel that my hearing is worse in a morning, gets better in the evening, and there have been times when I've been up in the very middle of the night where I felt my ability to hear subtle details was exceptional. But it's just a theory, I could be wrong about that.
  • In the ABX test, I tried to listen to a different excerpt for each of the 16 trials. In other words, I never used the same portion of the song twice.
  • For some trials, I got a strong feeling straight away and was able to answer relatively quickly. While for other trials I didn't get a strong feeling straight away and moved on to another random excerpt. Sometimes I moved onto a third random excerpt, but usually only one or two were sufficient
  • All the excerpts I listened to were approx. 2-4 seconds long, which I constantly "looped" by hitting the play buttons over and over. (I'm actually surprised there's no built-in loop feature. For me, hearing silence would destroy the test and would probably be irritating too.)
  • I did listen at a fairly high volume. As a musician, I often listen to things at relatively high volume so that I can hear fine details. I am aware that this is damaging to the ears, but to me it doesn't seem harmful and I think I know what my ears can tolerate. For example, if I'm walking down the street and an ambulance or motorbike is approaching, I always put my fingers in my ears because I know that a sudden loud noise like this would be very distressing and no doubt hurt my ears. But when I'm listening to music, it feels fine to ramp up the volume, so long as I gently warm my ears up, always starting at a low volume and increasing gradually (and then I would not listen at high volume for too long). Perhaps I am damaging my ears but this is what feels right to me, and I always listen to my body.
  • My method was always to listen to A once, then B once, and quickly decide which I felt was the WAV (ie which sounded better). Then I may listen to A and B again, but would promptly move on to X and Y and decide the same for them. By this point I would have some idea in my mind which two samples were the WAVs, and I would listen to all 4 again to confirm it.
  • Interestingly, the difference between samples was usually more apparent the very first time I heard them. I found that the more I listened to a pair, the more the perceived difference would diminish, and the more they started to sound identical. Thus, if I had not very quickly made any determination, I would move on to a fresh excerpt.
  • Some of the time I felt very strongly I could hear the difference, but other times, I was less certain. I do feel I could have gotten a better result if I'd taken the time to make sure I was certain every time. But since this was my first ABX test, I really just wanted to get through it fairly quickly and see how I did. If I do another, I will try to only answer when I'm sure.
  • In the next two points I will describe the actual differences I heard, but first I want to make a big disclaimer that the differences I heard are extremely subtle. The actual mix of parts and instruments sounds identical. There are no missing elements, not even the subtlest of harmonies, or reverb hiding quietly in one ear (as I had feared there might be). Every part is present and correct, which is great :) The only differences I heard were extremely subtle - to the degree that they are almost subconscious. I think that for most listeners, these differences would probably be negligible and this shouldn't be a big deal. It's not even a big deal to me, and I certainly won't be racing to re-encode my music collection. Though given the choice, I would prefer a codec which retained the subtle differences I heard.
  • As for where I hear the differences, they seem to lie in what I would call the "spatial" or "ambient" realm - so it's less about stuff like lead vocals, and more about the blend of all the quiet little bits of reverb in your left and right ears. Often there is just what I would call a "fuller spatial picture". For example, LAME will accurately translate the noises that are happening in the "center" and "on the right",  but the original WAV will "fill out" the space between center and right. So it sounds a little more like a "real space" with a "fuller sound".
  • Well, finally, let me try to describe the actual difference in sound quality. There are many ways I could describe it. One way is that the WAV simply sounds like it's coming from a more expensive speaker system. Another way I would describe it—and forgive me if this sounds like the kind of cliche you'd hear from computer-hating vinyl lovers (for the record, I hate vinyl and love digital audio!)— but to me the WAV sounds slightly more "wholesome", "natural" and "old fashioned". It seems "fuller" and "richer" and has slightly more of a "crackle" to it, like there's more subtle high-frequency texture to it. Strangely, it reminds me of the kind of sound I heard when I was growing up. I don't know whether that's vinyl, or on the radio, or what—but it just sounds more old fashioned and wholesome somehow. But above all, what stands out the most about the WAV is just a "feeling" - it just kind of "grabs" me more and is more engaging with more "power". It's like it touches my heart more. It also feels more relaxing somehow, and less stressful. Oddly, the WAV can even sound a little louder at times (I have tested the files in question and there is absolutely no difference in gain.)

  • bennetng
  • [*][*][*][*][*]
Re: Thinking of switching codec - need some advice please
Reply #24
The track I tested was Shola Ama - Much Love

Output:
DS : Primary Sound Driver
Hi, since TOS8 requires a double-blind test and a sample audio file so that others can examine it, can you upload a lossless snippet of this song for us to examine? A 30-second file is allowed, according to TOS9:
https://hydrogenaud.io/index.php/topic,3974.msg149482.html#msg149482

Unfortunatly this makes things more complicated for the OP.
I guess also with the straightforward procedure he will have a hard time finding something he can ABX.
In case he should really find someting we can have a look at the provided sample and, if the track peak value should give rise to the suspect that the overall playback chain will change volume, we can ask him to manually edit track replaygain values of the original (converted to FLAC) and the mp3 file and ask him to redo the ABXing using replaygain. We can help him to any amount with that in case it should be necessary.
Just grabbed the aac and vorbis files from Youtube and got some interesting track peaks (see attached screenshot). Just wanted to make sure it is not the cause of audible differences to avoid some useless or even harmful codec tweaking for developers.

the WAV sounds slightly more "wholesome", "natural" and "old fashioned". It seems "fuller" and "richer" and has slightly more of a "crackle" to it [snip] the WAV can even sound a little louder at times (I have tested the files in question and there is absolutely no difference in gain.)


  • Last Edit: 18 June, 2017, 11:58:44 AM by bennetng