Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: ABXing FLAC and MP3: Are there flaws with my methodology? (Read 7291 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

ABXing FLAC and MP3: Are there flaws with my methodology?

I'm a longtime lurker here, but a first time poster.  My situation is that I have about 1500 CDs worth of mp3s that I ripped from my collection around 2004-05 at CBR 128kbps using the Creative Mediasource software that came with the Zen Xtra I used to have.  I've learned more about mp3 since that time and have had this nagging regret that I didn't do more research when I did all that ripping in order to get better sound quality.  My long term plan has been to rerip everything to FLAC on an external drive and then spool off mp3s to LAME at a vbr to be determined later to replace my 128kbps files. 

However, I conducted a small experiment myself with a couple of song the other day.  Just because I had it handy on my laptop from burning some mp3 discs for work I picked an mp3 of "A Hard Day's Night".  Then I pulled the CD from my cabinets and made a FLAC of the same song. I figured I would use a comparison of these two tracks as a starting point for eventually determining at what bitrate I would ultimately rerip my collection. I pulled out my Yuin PK3 earbuds, which are what I use with my iPod, and fired up Foobar to do a blind comparison fully expecting it to be slam dunk easy for me to tell the difference between the FLAC and the mp3. It was way harder and the differences were far more subtle than I expected. I could probably train myself to differentiate them more reliably, but I was largely just guessing as to which was which.  I thought that it might be because I had chosen an older recording and that I might have an easier time of it with something more modern and dynamic with more stuff going on in it. I repeated my experiment with a Sigur Ros track, "Hoppípolla", and had similar results to what I had with "A Hard Day's Night".  Just about every time I thought I was hearing a killer flaw that might be the mp3 encoding revealing itself I heard the same flaw on the other track.  I tried some nicer Sennheiser headphones that I sometimes use at home and my performance in the ABX testing did not improve appreciably.

Anyway, I'm wondering if these results are something that would be expected for most people or if I've done something wrong.  My hearing is fine and probably better than average for a 37-year-old.  My laptop is a Dell Inspiron E1705.  I don't know any specifics about the sound card, but it's just the stock one that came with the computer.  Maybe I don't need to do all this reripping after all and should just chill with what I have.  Does 128kbps get a bad rap because people listen to it with a negative mindset and listen for flaws without realizing that the flaws are also present on the original recording? 

Also, if/when I do this what external hard drive would be the most reliable?  Would I be better served and could I make the process any faster by buying an external CD drive for ripping instead of using my laptop's drive?

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #1
I'm older than you (so my high frequency hearing is not nearly as good, although yours is not as good as an 18 year old's either).  This said, I've tried to ABX FLAC, WAV, etc. files back to good quality LAME mp3 encodes and NEVER been able to distinguish. In typical cases, a good mp3 should be transparent. The few times I think I detect problems, it turns out I can hear the same flaw in the original CD itself.  I think most people misunderstand the recommendation to maintain a FLAC (or other lossless) archive. It is generally NOT because it will sound lots (or any) better than an mp3. Rather, it allows one to create an archive that is lossless, and any future encodes to other lossy codecs will not require any transcoding from lossy to lossy.  Or for example, you can easily create small mp3 files for sticking on an ipod shuffle for exercising, or for playing in a noisy car environment, but have higher quality mp3's (that are larger) for playing in a quieter environment. If you rip to lossy to begin with, you can't really do all these things without transcoding from lossy to lossy (which you should avoid).

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #2
Anyway, I'm wondering if these results are something that would be expected for most people or if I've done something wrong.


You aren't doing anything wrong.  foobar2000's ABX option was designed to allow you to perform correct ABX tests.  I don't know about the results though as different people have different ears, different music, and different equipment.  I can point you to some older public listening test results that compared encoders at 128kbps.  Lame (which was version 3.97 at the time) performed pretty well at -V 5 (~130kbps VBR).  I think that 128kbps gets negative perception (without listening to it) because many people think that it is too low of a bitrate.  Most people don't actually conduct blind ABX tests; they simply make sound quality statements using previous negative dispositions, using what some other audiophool said, or they think that their "listening test" (which was nothing more than them switching back and forth in their audio software) was more than enough.  I know I have surprised many of my friends by sitting them down to conduct a blind ABX test and having them fail it at 128kbps mp3/AAC.  There was one friend who used to encode at nothing but 320kbps for "perfect quality."  I sat them down in front of their computer and had them conduct a blind ABX test comparing the lossless source and a FhG mp3 encoded at 96kbps CBR.  They failed.  Of course they said that I somehow messed the test up and refused to acknowledge the results.  Still, it just goes to show you that things like this can happen.

Also, if/when I do this what external hard drive would be the most reliable?  Would I be better served and could I make the process any faster by buying an external CD drive for ripping instead of using my laptop's drive?


I think you should be fine using your computer's internal drive so long as the results are secure (use EAC or dBpowerAMP) and match up with AccurateRip results (use EAC or dBpowerAMP).  I have never gone out and purchased an external optical drive just for ripping as I have always used either EAC or dBpowerAMP for my ripping needs.  Additionally, you shouldn't worry about ripping speed too much.  My tablet PC can rip CDs at about 15X while my desktop can do it at 20X.  Even with the added speed, both rip and encode a CD in a few minutes (less than 5).  The increase in ripping speed doesn't really improve the ripping time on my desktop.  What you want to focus on is the quality of your CD rips.  Make sure that the program you use (EAC is free, dBpowerAMP costs money but it is just as good but I think it is much easier to use) works with AccurateRip.  That way you can compare your CD rips to something.  I would stay away from using software such as Windows Media Player, iTunes, MusicMatch (it is old but I still come across people who use it), etc. since they don't work with AccurateRip and their "error correction" methods are horrible.  They will basically plow through any error found on a CD (due to a scratch, dust, fingerprint, etc.) without telling you.  Both EAC and dBpowerAMP will let you know that they came across an error.

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #3
1500 CDs is quite a large collection. Think a long time before re ripping your collection to FLAC, or be patient (ripping only 10-20 CDs a day). I only have a little over 200 CDs and re ripping my collection took forever.

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #4
While I don't remember my ABX results from when I tried it, from what I remember I am NOT able to pass an ABX test of FLAC versus 192 kbps CBR.  From my informal testing I seem to be able to hear all the way to 17K-18K.  I imagine my ABX result is fairly typical.  There's nothing wrong with your hearing - don't worry about it.  People claim that using an expensive sound card with an expensive pair of headphone will help you tell the difference between MP3 and FLAC.  In my experience such a statement is not true unless you have headphones with obviously deficient ($1 earbuds) sound.  Sound cards build into modern (last few years) laptops and computers really are quite good.

I'd rather not do a 128kbps ABX.  I'd don't want to face the reality of failing

As far as ripping to FLAC, I keep a combination of FLACs and MP3s.  To save space, I've used FLAC (and MP3s) to backup CDs I really like and do not want to lose.  For the rest of my CDs (the large majority if them) I've converted them to MP3s and have no plan to save them as FLAC.

I've had good experiences with western digital external hard drives.  For whatever reason it seems external hard drives have a higher failure rate than internal drives.  I would suggest you have at least two copies of all your music regardless of how you store your music.  I would suggest you have two separate drives instead of one mirrored one.

Hope that helps.

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #5
1500 CDs is quite a large collection. Think a long time before re ripping your collection to FLAC, or be patient (ripping only 10-20 CDs a day). I only have a little over 200 CDs and re ripping my collection took forever.


At this point, I'm honestly not so sure that what I would gain would necessarily be worth my effort.  When I did my ripping in 2004-2005 I did about 10-15 discs a day for three plus months.  I would actually be ripping 1,600 plus discs if I do it again.  I also worry a little about backing up.  Right now I have everything on an external drive that I use for day to day stuff and I'm backed up on a second external drive that only gets used when I'm copying files to it.  My library fills up about 82GB.  I can do a complete backup from scratch in about 4-5 hours.  I could use the same system with FLAC, but it would take much longer to copy the files over. 

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #6
While I don't remember my ABX results from when I tried it, from what I remember I am NOT able to pass an ABX test of FLAC versus 192 kbps CBR.  From my informal testing I seem to be able to hear all the way to 17K-18K.  I imagine my ABX result is fairly typical.  There's nothing wrong with your hearing - don't worry about it.  People claim that using an expensive sound card with an expensive pair of headphone will help you tell the difference between MP3 and FLAC.  In my experience such a statement is not true unless you have headphones with obviously deficient ($1 earbuds) sound.  Sound cards build into modern (last few years) laptops and computers really are quite good.

I'd rather not do a 128kbps ABX.  I'd don't want to face the reality of failing

As far as ripping to FLAC, I keep a combination of FLACs and MP3s.  To save space, I've used FLAC (and MP3s) to backup CDs I really like and do not want to lose.  For the rest of my CDs (the large majority if them) I've converted them to MP3s and have no plan to save them as FLAC.

I've had good experiences with western digital external hard drives.  For whatever reason it seems external hard drives have a higher failure rate than internal drives.  I would suggest you have at least two copies of all your music regardless of how you store your music.  I would suggest you have two separate drives instead of one mirrored one.

Hope that helps.


My hearing seems to top out in about the same place as yours.  I get a little bit of a sensation over 18khz, but I wouldn't say that I'm necessarily conscious of actually "hearing" anything up there.  Considering I saw Maiden, Metallica, Queensryche and Ozzy in high school without ear plugs I seem to be doing OK in that regard.  It's just what when I did all this ripping I didn't know a damned thing about mp3 and just used the default settings in Creative Mediasource and I didn't learn that I could have done better for myself until well after the fact.  I actually only just ditched my Zen Xtra for an iPod Classic a few months ago because I was sick of not having functional playlists due to the fact that I had upgraded the hard drive to 100GB and the Zen Xtra firmware gets a little sketchy and your playlists stop working once your library gets a little bit over 60GB.  The sound quality of the iPod is all ready a noticeable upgrade from the Zen Xtra, even using the exact same files. 


ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #7
At this point, I'm honestly not so sure that what I would gain would necessarily be worth my effort.


The benefit would be to never rip your audio CDs again.  Rip them to FLAC once and you are done.  You could then encode your FLAC files (while not deleting them from your computer) to ANY other format and that would produce the exact same results as if you were to go back and re-rip the CDs.  Rip once, rip right.  I used to re-rip my CDs about once every 5 years and, as my collection drastically increased, the ripping process would be come a pain.  I decided to rip my CDs to Apple Lossless (I have an iPod, AppleTV, and I use iTunes for my PC music needs) using dBpowerAMP back in 2007.  dBpowerAMP is a secure ripper and all of my results were checked with AccurateRip.  Hence, I know that each one of my rips is just like the source CD (and lossless is bit-for-bit the same as the source CD).  Now I can simply setup my computer over the weekend to encode my lossless files to a new lossy format.  I could even convert my ALAC files to a new lossless format.

I really feel that the benefits of ripping to lossless (whether it be FLAC, WavPack, WMA lossless, Monkey's Audio, or TAK) greatly outweigh the con (time spent ripping).  I have about 1800 CDs in my library and I know that I will never have to rip them again as I have one 2TB USB2.0 hard drive containing the files, a backup 1TB USB2.0 hard drive, and I burned every ALAC file onto dual-layer DVDs.  I don't think I needed to go through all of that as my primary (day-to-day) 2TB USB2.0 hard drive has never had any hardware problems *knocks on wood*.

I used to never understand the idea behind lossless until I realized that it could be used as a digital archive and lossless really gives you the freedom to do whatever you want.  I still listen to lossy files (only) but my lossless files are there.  I recently benefited from my lossless files as I conducted ABX tests between the iTunes AAC encoder, Nero AAC encoder, and Lame mp3 encoder.  My lossy library was reduced by a few GB by me switching to Nero AAC at -q0.5.  My tablet PC (1.5GB DDR RAM with a 1.6GHz Pentium M [Centrino]) takes about 36 hours to encode all my lossless files to a new lossy encoder/setting.  I setup foobar2000 to encode all my files, hit convert, walk away for a few days, and come back to see that it is finished.  This is much better than having to sit at my computer inserting and ejecting CDs.

 

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #8
Try something else other than Rock and post the results.

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #9
Quote
I tried some nicer Sennheiser headphones that I sometimes use at home and my performance in the ABX testing did not improve appreciably.


This usually ends up being the case with most people. They think that can hear a difference, but when it comes time to perform as series of ABX tests they find out that the placebo effect had just been consuming them. What they think they were hearing is actually not true. I had this same problem when I first started doing ABX tests, but came to except what I thought I could hear wasn't actually there and I was just guessing. 
budding I.T professional

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #10
Above a certain threshold (outputs without considerable levels of noise or hiss) lossy compression artifacts become actually less and less audible with better equipment, in contrast to what most "audiophiles" think. The reason is that lossy codecs are optimized for a perfectly flat frequency response, so a MP3 will sound best with very good equipment. Inferior equipment with little spikes and boosts at certain frequencies can unmask compression artifacts instead.

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #11
From what I know (certainly some on this board have much more 1st hand knowledge), the things that ok-but-not-great headphones fail on are not the things that lossy compression fails on, so super-fi equipment is not needed to detect the artifacts.

On the drive, an external CD drive might be good because the multiple reading rippers (like EAC) can be tough on a drive.  Internal laptop drives are likely not as robust and certainly more expensive to replace than an external one.



ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #12
Try something else other than Rock and post the results.


Truth to tell I don't have very much in the way of non-rock music.  I have a ton of prog.  What classical music I have was ripped at 192 because I somehow got this idea, possibly not true, that the more instruments were in the mix the higher bitrate I needed and most of my jazz leans towards older recordings that for whatever reason have always seemed particularly fantastic to me on either of my mp3 players.  Are there any particular tracks that someone might suggest as being particularly difficult to capture?

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #13
On the drive, an external CD drive might be good because the multiple reading rippers (like EAC) can be tough on a drive.  Internal laptop drives are likely not as robust and certainly more expensive to replace than an external one.


That's exactly why I was thinking of getting an external CD drive.  I just don't want to prematurely kill the one in my laptop.


ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #14
At this point, I'm honestly not so sure that what I would gain would necessarily be worth my effort.


The benefit would be to never rip your audio CDs again.  Rip them to FLAC once and you are done.


I think that I probably will undertake this project eventually just as you described, but at a slower pace than I was planning since my experience so far with ABX testing has greatly lessened this nagging urgency that I need better mp3 files in order not to be badly cheating myself on my music listening.  It's kind of a relief.  However, I also assume that options for lossless playback are only going to expand and I still will want to experiment with different bitrates and lossy codecs so maybe I'll work on just ripping 3-5 discs per day when I get my pieces in place.

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #15
I found a track that I can nail every time.  With the 128kbps mp3s there is definitely some clipping and/or static on eighties rockin' type guitars that I can very easily detect, such as Trevor Rabin's signature riff from "Owner Of A Lonely Heart".

foo_abx 1.3.3 report
foobar2000 v0.9.6.2
2009/03/15 17:34:08

File A: C:\Documents and Settings\Chris\Desktop\Experiment\01 Owner Of A Lonely Heart.mp3
File B: C:\Documents and Settings\Chris\Desktop\Experiment\Owner Of A Lonely Heart.flac

17:34:08 : Test started.
17:36:16 : 01/01  50.0%
17:36:48 : Trial reset.
17:40:13 : 01/01  50.0%
17:41:20 : 02/02  25.0%
17:42:40 : 03/03  12.5%
17:44:04 : 04/04  6.3%
17:45:07 : 05/05  3.1%
17:46:52 : 06/06  1.6%
17:48:33 : 07/07  0.8%
17:49:10 : 08/08  0.4%
17:49:47 : 09/09  0.2%
17:50:25 : 10/10  0.1%
17:50:47 : 11/11  0.0%
17:51:13 : Test finished.

----------
Total: 12/12 (0.0%)

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #16
I somehow got this idea, possibly not true, that the more instruments were in the mix the higher bitrate I needed.


Yes, it's not true.

Are there any particular tracks that someone might suggest as being particularly difficult to capture?


I think nearly all material are difficult at 128.. also the result are vary greatly by encoder AND the listener.

I found that, to me, Rock is one of the less annoying material at 128, the artifact are there but not really ruined the sound that much.

The ugliest to me are any type of orchestra: Violin sound smoothed and warped (especially at low volume) and piano/acoustic album: pre-echo and flutter ruined it for me.

I believe Electronic should be pretty hard as well but I'm not really a fan of it so I don't have much experience about it.

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #17
On the drive, an external CD drive might be good because the multiple reading rippers (like EAC) can be tough on a drive.  Internal laptop drives are likely not as robust and certainly more expensive to replace than an external one.


I don't use laptop drives for ripping audio as I have seen too many of them fail way too fast - even when not used rigorously. I would use a desktop for ripping audio if I were you.

ABXing FLAC and MP3: Are there flaws with my methodology?

Reply #18
On the drive, an external CD drive might be good because the multiple reading rippers (like EAC) can be tough on a drive.  Internal laptop drives are likely not as robust and certainly more expensive to replace than an external one.


I don't use laptop drives for ripping audio as I have seen too many of them fail way too fast - even when not used rigorously. I would use a desktop for ripping audio if I were you.


This leads into my next question.  Has anyone ever used this thing or something similar to rip a bunch of CDs:  http://www.amazon.com/Acronova-DupliQ-Auto...9291&sr=8-1 ?

I've researched it a little bit and I get the impression that it might work with the dBpoweramp batch ripping software.  If something like this works then I bet it would save me a lot of time and I could always ebay the thing when I'm finished to get some of my money back out of it.