Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: MP3 Listening Test at 128 kbps (Read 28753 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MP3 Listening Test at 128 kbps

This should be the thread for collecting samples to be used in the upcoming MP3 listening test at 128 kbps. Please upload everything that you think it might be useful here or also recommend any samples that were used in previous tests. Since we have 5 contenders + a low anchor, I think it is best to limit the number of samples to 12.

Eagerly waiting for your suggestions...

MP3 Listening Test at 128 kbps

Reply #1 –
It wouldn't be fair to suggest those samples targeting specifically at weaknesses of Lame I am so much used to (though they are problematic for other encoders too).
Moreover it's not useful to test annoying sounding samples with a 128 kbps mp3 test.

Taking this into account my suggestions are:

a) castanets as a natural instrument pre-echo problem (not annoying at this bitrate at least to my ears)
b) Suzanne Vega's Tom's Diner as a potential tonal problem.
lame3995o -Q1.7 --lowpass 17

MP3 Listening Test at 128 kbps

Reply #3 –
Difficult sample for 128 kbps MP3, especially the synth brass.

MP3 Listening Test at 128 kbps

Reply #4 –
Quote from: naturfreak on
Difficult sample for 128 kbps MP3, especially the synth brass.


That sample does sound bad at -V 5 if you crank up the volume, or use ReplayGain.
"I never thought I'd see this much candy in one mission!"

MP3 Listening Test at 128 kbps

Reply #5 –
I also got a couple suggestion tracks from Vangelis. I have made a couple 30 sec samples from the Blade Runner soundtrack, which is mix of Electronic Music and a Movie Score, which I find it usefull for testing out a lossy codec.

Also Replica by Fear Factory struggles on must mp3 encoders, I have tested at 128kbps or at 130 VBR.
"I never thought I'd see this much candy in one mission!"

MP3 Listening Test at 128 kbps

Reply #6 –
Another challenging sample for 128 kbps MP3

MP3 Listening Test at 128 kbps

Reply #8 –
Quote from: mr budzone on
I recommend this one

(alternate slow link if attached file somehow stops working)
Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.4
2008/07/15 00:15:51

File A: C:\Downloads\finalfantasy.flac
File B: C:\Temp\Transcodes\Samples\finalfantasy.mp3

00:15:51 : Test started.
00:16:27 : 01/01  50.0%
00:16:34 : 02/02  25.0%
00:16:43 : 03/03  12.5%
00:16:50 : 04/04  6.3%
00:16:58 : 05/05  3.1%
00:17:13 : 06/06  1.6%
00:17:29 : 07/07  0.8%
00:18:03 : 08/08  0.4%
00:18:09 : 09/09  0.2%
00:18:17 : 10/10  0.1%
00:18:18 : Test finished.

 ---------- 
Total: 10/10 (0.1%)

A very good example, that MP3 struggles at harpsichords.
"I never thought I'd see this much candy in one mission!"

MP3 Listening Test at 128 kbps

Reply #9 –
(Trash-can this if it is going too far off topic)

All this brings up an interesting point.  Could you not (relatively) easily stack the deck for or against one particular encoder through nothing more than sample selection?
I stress I am not trying to accuse anyone of this - but it does sound like it is entirely plausible that such a feat could be accomplished.
Creature of habit.

MP3 Listening Test at 128 kbps

Reply #10 –
Quote from: naturfreak on
Difficult sample for 128 kbps MP3, especially the synth brass.
Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.4
2008/07/15 00:56:23

File A: C:\Downloads\Vangelis___Chariots_of_Fire__edit_.flac
File B: C:\Temp\Transcodes\Samples\Chariots of Fire (edit).mp3

00:56:23 : Test started.
00:56:33 : 01/01  50.0%
00:56:39 : 02/02  25.0%
00:56:47 : 03/03  12.5%
00:56:56 : 04/04  6.3%
00:57:03 : 05/05  3.1%
00:57:10 : 06/06  1.6%
00:57:20 : 07/07  0.8%
00:57:28 : 08/08  0.4%
00:57:35 : 09/09  0.2%
00:57:45 : 10/10  0.1%
00:57:50 : 11/11  0.0%
00:57:54 : 12/12  0.0%
00:57:56 : Test finished.

 ---------- 
Total: 12/12 (0.0%)

The backgroud noise on this sample sounds like it has a bad case of warbling and precho. I think the analog hiss in background makes the track harder to encode, I encounted something like that with Black Sabbath's Iron Man, which also sounds bad at 128kpbs, but it is not as bad as this sample.
"I never thought I'd see this much candy in one mission!"

MP3 Listening Test at 128 kbps

Reply #11 –
Quote from: Soap on
....All this brings up an interesting point.  Could you not (relatively) easily stack the deck for or against one particular encoder through nothing more than sample selection?....

Yes, that's a problem. Nobody knows to what extent the outcome of a listening test depends on the selection of the samples. I'm convinced that many people take the results in a much too serious way as a statement on encoders' quality. There's also the chance of misinterpretation of the statistical analyses. Many people take it a as a degree of certainty whether a certein encoder is better than another. But it just tells about the degree of certainty of judging the test results of the samples tested. It can't tell about to what degree the results of the samples tested can be generalized to a statement about general encoders' quality which is the real interesting question you brought here. I really hate the 'zoomed view' which IMO increases a wrong understanding of test results.

Anyway, nobody can work miracles. And any listening test contributes towards experience on encoders' quality. This is worth something. The experience from a specific test just shouldn't be taken too much as a statement on general quality of the encoders tested.

Gross systematic errors in sample selection can be avoided by a simple procedure like 'don't use more than 1 sample from 1 person'.
lame3995o -Q1.7 --lowpass 17

MP3 Listening Test at 128 kbps

Reply #12 –
Thank you, halb27.
I guess the idea of randomly selecting samples from a large music library has the problem of likely not hitting any problem samples and failing to show any difference between encoders?

Can even a casual inference be drawn that there is likely a slight bias in these tests against popular encoders as they are the ones most likely to have had their problem samples identified?
Creature of habit.

MP3 Listening Test at 128 kbps

Reply #13 –
Quote from: /mnt on
...

That artifical infrabass on the Vangelis Main Titles is pretty brutal! 

http://martel.ic.cz/bordel/clavicula_sample.flac (5.4MB)

If you can get the treble on this through your audio chain, you definitely will not get it through LAME's lowpass. It is somewhat similar to that Final Fantasy sample, although I suspect the latter was played on a keyboard (what an unnatural sound), not a guitar.

MP3 Listening Test at 128 kbps

Reply #16 –
Has any thought been given to getting samples of "popular" music?  I mean music that is in the "Top 10" or "Top 20" downloads on iTunes or Amazon over the past month in each of several genres (Pop, Rock, Alternative, Rap/R&B, Country, Latin).

To me, that would be more applicable for the general listening public, than a quick sample from a relatively obscure artist that maybe 1/100 % of the world listens to and buys their music.

Artists such as Usher, Ne-Yo, Katy Perry, Leona Lewis, Coldplay, Kanye West, Carrie Underwood, Taylor Swift, Rihanna, Natasha Bedingfield, Sugarland, We The Kings, Secondhand Serenade, Jon Mayer, OneRepublic, etc.

Any artifacts at 128kbps with "popular" music is more likely to be noticed by the general listening public, and thus be a more pressing need to rectify, I would think.

MP3 Listening Test at 128 kbps

Reply #17 –
Quote from: WonderSlug on
Has any thought been given to getting samples of "popular" music?  I mean music that is in the "Top 10" or "Top 20" downloads on iTunes or Amazon over the past month in each of several genres (Pop, Rock, Alternative, Rap/R&B, Country, Latin).


I think that the problem is that pop, country, and rap/r&b tends to be rather easy to encode.  For example, I can rip and encode an Eminem track (the song "Bad Influence" is an example) and I cannot readily ABX it at even 128kbps VBR AAC (Nero or iTunes).  However, I can ABX "Love Lost In A Hail of Gunfire" by Bleeding Through at 128kbps VBR AAC (both Nero and iTunes).  I can even ABX said song at -V 2 using Lame 3.97/3.98 to a certain extent.  After listening to it enough to judge 8 different times, my ears begin to get a little tired from the wall of metal sound produced in that song.

I could be wrong though.

Edit:  This public listening test is also to be conducted here on Hydrogenaudio, I don't think that many people here on HA listen to the county's (ie the U.S.) most popular music.  If this test was going to be conducted by the mass public then maybe.  Even then, the mass public would probably be fine with nearly any modern mp3 encoder you throw at them at 128kbps VBR.

MP3 Listening Test at 128 kbps

Reply #18 –
Quote from: IgorC on
Spill the blood
http://www.hydrogenaudio.org/forums/index....ost&id=3453

And also it will be interesting to see old sample waiting
http://www.rarewares.org/test_samples/Waiting.wv

SLAAAAAAAAAAAYYYYYYYYYEEEERRR \m/. Sorry I could not resist  . The begaining of the track is noticable at -V 5.

Quote from: kornchild2002 on
Quote from: WonderSlug on

Has any thought been given to getting samples of "popular" music?  I mean music that is in the "Top 10" or "Top 20" downloads on iTunes or Amazon over the past month in each of several genres (Pop, Rock, Alternative, Rap/R&B, Country, Latin).


I think that the problem is that pop, country, and rap/r&b tends to be rather easy to encode.  For example, I can rip and encode an Eminem track (the song "Bad Influence" is an example) and I cannot readily ABX it at even 128kbps VBR AAC (Nero or iTunes).  However, I can ABX "Love Lost In A Hail of Gunfire" by Bleeding Through at 128kbps VBR AAC (both Nero and iTunes).  I can even ABX said song at -V 2 using Lame 3.97/3.98 to a certain extent.  After listening to it enough to judge 8 different times, my ears begin to get a little tired from the wall of metal sound produced in that song.

I could be wrong though.



I completely agree, most of the "Radio Friendly Unit Shifter" tracks, can mostly encode without any problems. Also alot of Pop, Rap and R&B, is likely to encode at the below estimated VBR bitrate.
"I never thought I'd see this much candy in one mission!"

MP3 Listening Test at 128 kbps

Reply #19 –
A lot of that depends on whether or not the audio frames can be encoded almost entirely in M/S stereo or whether most of the frames are done in L/R stereo.

I have "Top 40" music that if encoded via the -V 4 setting, will encode at below 160kbps (which is the target for V4).  In this case, this particular music has a lot of frames that are encoded with M/S, about 90% or so.  For those, I usually get around 145 or 152 kbps.

However, I also have other "popular" music, that if also encoded via -V4, will encode as high as 170 or even 180kbps VBR for a few tracks.  In this music, a lot of the frames are encoded L/R stereo.  Examining the encoding closely, there tends to be either a near-even split, such that about 45% LR and 55% MS, or even mostly LR, like 68% LR and 32% MS.

Now, if the vast majority of "Top 40" music can be encoded at -V4 or -V5 with perceptual transparency for the vast majority of listeners, I would question the cost/benefit of spending the time and effort to try to gauge the issues with a few problem samples in obscure music.

In other words, if 95% of the music that 'regular' people listen to can be encoded at -V4 or -V5 and be transparent for 95% of the listeners, why spend all this effort on the 5% of music and people that are on the fringe?  For those, LAME may never be good enough even at 320kbps CBR, and it would be better to move to either lossless, or a superior lossy algorithm like AAC or MPC.

For me, -V3 has always been transparent for me, in all the music I listen to, even Classical and soundtrack/ambient.  I moved to 3.98 from 3.97 because of the somewhat faster encoding speed, not because of any encoding quality improvements, even though supposedly the equivalent -V mode yields a slightly higher bitrate with 3.98 than 3.97.

Another thing that comes to mind is what about a complete statistical analysis of the encoding process of 3.97 versus 3.98?  I mean take a particular WAV/FLAC/TAK/WavPack file and encode it in both 3.97 and 3.98 at the various -V modes (-V0, -V1, -V2, ..., -V5).  Examine in detail, how the frames are encoded, as in how many are encoded at 96kbit, 112kbit, 128kbit, ..., 320kbit.  Then examine the percentage that are encoded L/R and M/S stereo.  See what, if any, changes occur in 3.98 from 3.97.  Next, do the listening test of such, and if 3.98 sounds better than 3.97, you can look at the statistical breakdown and make an educated guess maybe at where 3.98 did it better.  Maybe it encoded some frames at 224kbit in 3.98 that were encoded at only 160kbit or 192kbit in 3.97.  Also, maybe more of these frames were able to be done in M/S versus L/R in 3.98 over 3.97, thus giving more space in the bit reservoir for higher encodings.

MP3 Listening Test at 128 kbps

Reply #20 –
Quote from: /mnt on
I completely agree, most of the "Radio Friendly Unit Shifter" tracks, can mostly encode without any problems. Also alot of Pop, Rap and R&B, is likely to encode at the below estimated VBR bitrate.


I am glad someone agrees with me.  One of my roommates encodes all their audio at 192kbps VBR AAC with iTunes and the majority of their content is either radio friendly rock (the light stuff) or hip-hop/r&b/rap.  I told them to conduct a test and that they probably wouldn't be able to differentiate between a 128kbps VBR AAC file and the source CD.  Instead, they decided to encode at that bitrate just because they could and then they complain about not being able to fit very many songs on their 3G 8GB iPod nano.

Anyway, I think we should put up a thrash/hardcore/metal song (maybe two) to be tested.  Slayer's "Spill The Blood" would be good and I also think that "Love Lost In A Hail Of Gunfire" would be good as well.  I know that with Lame 3.97, I could readily ABX between the source lossless file and a -V 5 encoding of it.  I would provide a sample but I am currently not at my computer where my music is stored.

MP3 Listening Test at 128 kbps

Reply #21 –
Quote from: WonderSlug on
To me, that would be more applicable for the general listening public, than a quick sample from a relatively obscure artist that maybe 1/100 % of the world listens to and buys their music.

Following your idea, you could as well restrict the hardware used in the test as majority of people will be using $50- headphones, a DAP/mobile/cheaper soundcard etc... and turn the test into a mediocrity feast. 
You could then conclude that 128 kbits (regardless of codec) is already good enough for majority of people... which is not exactly the goal of the test, I suppose.

MP3 Listening Test at 128 kbps

Reply #22 –
Quote from: Martel on
a DAP/mobile/cheaper soundcard etc... and turn the test into a mediocrity feast.

That won't be a test with conditions of real life. Sometimes (often, not sometimes) artifacts are more audible on cheap hardware.

MP3 Listening Test at 128 kbps

Reply #23 –
This track always sounded obviously compressed to me when I heard it on the radio and other people's music collections. However, my 160kbps Nero AAC encoding sounds fine to me, though I haven't tried ABXing with the original.

I guess the sound of these drums is just difficult to replicate