Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Request for a listening test (Read 10715 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Request for a listening test

In the last few months I have read many concerns about using MPC due its lack of support by the author and the recent improvements of the newer encoding formats (AAC above all).
    As a member of this community and lover of MPC quality I must admit that this concerns are true. Unsupported software lead to useless data in a couple of years.
    Just for curiosity and to have to force to abandon THE KING (MPC) I request to the gurus of this board to setup and run a listening test  between MPC and the relevant others.
    I think the result of this test would help many of us in our choice for the codec to use in our next encoding.

    Thanks in advance. The discussion is open !

    Hecatombles

Request for a listening test

Reply #1 –
I see no need for such a test. The only thing you have to ask yourself is:what portable player do I want and what formats does it support?

If all you want are quality files,then just use the recommended quality setting for the format needed for the player-for lame  mp3 aps, for wma-some higher bit rate 2 pass vbr, for aac nero-transparent profile, for quicktime/itunes aac-192 kbps, for vorbis q5 or q6. That basically ends the need for any test.
you will make mp3's for compatibility reasons.

Request for a listening test

Reply #2 –
The discussion is not on the convenience of the format for using it on the portables.
Many of us are using MPC for arkiving. I need only to find the answer to the question: is the MPC quality still the best for lossy arkiving or have the recent innovations obfuscated it in some way ? If yes: how much ? Only a listening test could answer this question.

  Hecatombles

Request for a listening test

Reply #3 –
Ask Garf or search around. He tried running such a test but found it was futile due to the lack of usable results. Some people here struggle enough with the 128k tests.

Request for a listening test

Reply #4 –
Quote
Many of us are using MPC for arkiving. I need only to find the answer to the question: is the MPC quality still the best for lossy arkiving or have the recent innovations obfuscated it in some way ? If yes: how much ? Only a listening test could answer this question.

But most other formats don't focus on higher bitrates at all. Currently, their improvements seem to be almost exclusively for low bitrates. I don't think MPC has been surpassed in regards of being the "most often transparent" lossy codec for archival needs.

Request for a listening test

Reply #5 –
Dream on.

Request for a listening test

Reply #7 –
Quote
MPC Q4 won Roberto's 128kbit/s listening test

AAAAAAAAAARRRRRRRRRRGGGGGGGGGGGGGGGHHHHHHHHHHHHHHH

Request for a listening test

Reply #8 –
Quote
MPC Q4 won Roberto's 128kbit/s listening test

You seem to have missed this part:

One codec can be said to rated better than another codec with 95% confidence if the bottom of its line segment is at or above the top of the competing codec's line segment.
Happiness - The agreeable sensation of contemplating the misery of others.

Request for a listening test

Reply #9 –
Quote
Dream on.

Yeah, it would be a tough test. Many untrained listeners already struggle at ~128 kbps. If the test doesn't focus on some notorious killer samples, only people with good ears and training will hear the differences. But, on the other hand, some codecs have specific problems that can't be solved simply by throwing more bits at a sample, for instance, default Ogg Vorbis doesn't improve on some problem samples anymore above a certain bitrate. It could also be the case with AAC; to my knowledge, high bitrate encoding efficiency was never seriously optimized there as well. Only a listening test would show.

Request for a listening test

Reply #10 –
Quote
Quote
MPC Q4 won Roberto's 128kbit/s listening test

AAAAAAAAAARRRRRRRRRRGGGGGGGGGGGGGGGHHHHHHHHHHHHHHH

I feel your pain
flac > schiit modi > schiit magni > hd650

Request for a listening test

Reply #11 –
I don't see the need for such a test. MPC would win hands down, that is, if anyone could actually tell a difference at those bitrates. There is no mystery here.

My own personal opinion is that the next best option, is AAC. It may even be a better option, depending on what you are looking for. Vorbis lovers, hold your flames.

Now that it seems that Nero is gapless, I'm looking at making the switch. The lack of good seeking, and the lack of any hope of hardware support with MPC is enough to make me look real hard at AAC.

But a listening test at Q7? That's really pointless. We already know the winner.
flac > schiit modi > schiit magni > hd650

Request for a listening test

Reply #12 –
I propose that if anyone "requests to the gurus of this board to setup and run a listening test", he first has to prove that it is humanly possible to actually participate in such a test. So as soon as you provide, say, ABX results proving you (or anyone else, let's not be nitpicking) can actually hear the difference between a reference, an mpc q7, lame ape and aac at around 200kbps or whatever else you're requesting, then discussion can begin.

Quote
AAAAAAAAAARRRRRRRRRRGGGGGGGGGGGGGGGHHHHHHHHHHHHHHH

I feel for you, Roberto.

Request for a listening test

Reply #13 –
Quote
So as soon as you provide, say, ABX results proving you (or anyone else, let's not be nitpicking) can actually hear the difference between a reference, an mpc q7, lame ape and aac at around 200kbps or whatever else you're requesting, then discussion can begin.

Even for MPC this has been done before, see for instance this thread. There are also existing problems for LAME alt-presets, various AAC implementations and for Ogg Vorbis even at bitrates higher than 200 kbps.

Request for a listening test

Reply #14 –
Quote
We already know the winner.

Careful with that, buddy. Everybody knew Xing would lose on the MP3 test (me included, I admit) :B

Request for a listening test

Reply #15 –
Quote
Quote
MPC Q4 won Roberto's 128kbit/s listening test

AAAAAAAAAARRRRRRRRRRGGGGGGGGGGGGGGGHHHHHHHHHHHHHHH



You should cut and paste your usual response, every time someone says there was a winner in that test.

Request for a listening test

Reply #16 –
Quote
Quote
So as soon as you provide, say, ABX results proving you (or anyone else, let's not be nitpicking) can actually hear the difference between a reference, an mpc q7, lame ape and aac at around 200kbps or whatever else you're requesting, then discussion can begin.

Even for MPC this has been done before, see for instance this thread. There are also existing problems for LAME alt-presets, various AAC implementations and for Ogg Vorbis even at bitrates higher than 200 kbps.

On the same sample? The only one I know of that can be done with is the castanets sample and maybe some other extreme impule clips.

Request for a listening test

Reply #17 –
Quote
Quote
We already know the winner.

Careful with that, buddy. Everybody knew Xing would lose on the MP3 test (me included, I admit) :B

I personally feel that Xing would have done much worse if the 41_30, Layla, Castanets and Fatboy samples had been used - Xing's lack of short blocks kills certain samples, while on other samples Xing’s sounds decent to OK. Personally, I don't think Xing is as horrible at encoding most clips as people make it out to be. Even so, the fact that it murders high frequencies and lacks short blocks would lead me to use Gogo or FhG over Xing any day. Just my 2 cents.....

Request for a listening test

Reply #18 –
Quote
Even so, the fact that it murders high frequencies and lacks short blocks would lead me to use Gogo or FhG over Xing any day. Just my 2 cents.....

What I conclude from the test is that apparently that's a much more reasonable tradeoff than previously thought. Problem samples are only just problem samples - they're not representative, and on a representative sample apparently Xing is simply good compared to Gogo and FhG.

Request for a listening test

Reply #19 –
Quote
If the test doesn't focus on some notorious killer samples, only people with good ears and training will hear the differences.

In my test I found that even people with 'good ears and training' had very serious problems, to the extent that one started arguing that it had to be allowed to exchange information about the problem parts in the clips - which I certainly wasn't going to allow.

Every codec fell through for someone, and I didn't see any particular trend like MPC being 'more transparent' (  ) than the others.

I fear the only way to get a significant result is to pick known very hard problem samples. But what would be the meaning of the result?

Request for a listening test

Reply #20 –
I have carefully read your posts and one conclusion seems to rise over you opinions: if even the most trained people in ABX testing would find HARD to discover the differences in the overall quality of the modern codecs versus MPC Q7 MPC is not worth for arkiving anymore.
If no one could easy find a difference.. there is no difference, isn't it !? The answer is here ! so we can abandon MPC and we can say goodbye to Frank Klemm and his unsupported MPC.

This seems too easy as answer and the answer was not ABXed.

Citay Wrote:

> If the test doesn't focus on some notorious killer samples, only people with good ears and training will hear the differences

I think the results coming from this test should be:

1) Is there any audible differences between so called "general samples" encoded with different codecs at this high bitrate ?

2) Regarding to Problematic samples, transients, echoes etc, how is the different codecs behaviour ? In this way we will find specific defects of the different codecs and the particular attitude of one codec  toward a kind of music.

With this set of answers I am sure anyone will be able to chose to abandon the idea of MPC as the only codec for lossy arkival purpose or not.

  Your opionions are again wellcome.

  Hecatombles

Request for a listening test

Reply #21 –
In my opinion (based on my experience), such test will only lead to statistically valid results if it's performed by hundreds of people.

The reason is that, no matter if they do spot the difference or not, it will probably be something so insignificant that the score will be 4.8 or 4.9 in that sample. In the end, all codecs will be ranked "up there" in the plots and there will be no conclusion to be made. My bet is that the confidence margins would all overlap, unless you use countless listeners (the higher the number of listeners, the smaller the margins, or so it seems for me).

Request for a listening test

Reply #22 –
IMHO, when using codecs/settings aimed at transparency, the right question to ask is not "which codec sounds more transparent at the given samples", but "which codec has less problem samples (i.e. sounds transparent more often than others)". This just means "which codec is more safe to use".

The sad thing is that this question is not any easier to answer

-Eugene
The  greatest  programming  project of all took six days;  on the seventh  day  the  programmer  rested.  We've been trying to debug the !@#$%&* thing ever since. Moral: design before you implement.

Request for a listening test

Reply #23 –
Using Q7 is really to high a bitrate to be of any use, and as we all know, you can't have something be more transparent. It is either transparent, or it isn't.

So, the question is, which codec produces transparency at the lowest bitrate (on average), and has the fewest and least annoying problem samples at that setting.

My understanding is that MPC at q5 is the leader in this category. I'm not very good at picking out artifacts, but I seem to be more sensitive to pre-echo. For this reason I choose MPC and/or  AAC over Vorbis and MP3 whenever possible.
flac > schiit modi > schiit magni > hd650

Request for a listening test

Reply #24 –
Quote
With this set of answers I am sure anyone will be able to chose to abandon the idea of MPC as the only codec for lossy arkival purpose or not.

It may be near-impossible to arrange a useful listening test at this high bitrate to answer your question.

But it certainly is possible for certain individuals to hear a difference between various lossy codecs on certain samples at high bitrates.

So mpc is certainly not about to be abandoned by people who hear artifacts in other codecs at high bitrates, but not mpc.


As for the question of "which codec fails least often?", there are threads which begin to answer this - just count the number of known problem samples for each codec and you'll get some idea. Unfortunately this count is biased by the number of people who use each format, and how critical they are.

It seems to me that no codec is perfect, and that people with very exceptional ears/brains can learn to recognise the faults in all codecs, even at "relatively" high bitrates, on at least some samples.

This isn't relevant to most people in everyday use.

Cheers,
David.