Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Multiformat@128kbps listening test - FINISHED (Read 182422 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Multiformat@128kbps listening test - FINISHED

Reply #150 –
Oh I forgot to say that I want to thank you nevertheless (see my last post), Roberto!

In the next listening we should use more songs with lower than average bitrates.

Big_Berny

Multiformat@128kbps listening test - FINISHED

Reply #151 –
Quote
Quote
My point is if they (minidiscers) claim that their MD hardware encodes better, then we should consider their claim. Similar to how we select the best encoder for other codecs in your test.

consider and then dismiss, if there is no proof for the claim, other than general subjective opinions.
if there is (semi-) scientific proof, it will will be gladly accepted.

guess what's gonna happen.

Well, they're the ones with the hardware.

Some one could easily record all the clips to their MD recorder via a digital link (from a non-resampling sound card), and then copy them all back into a PC. Then the three versions of each clip (original, software encoded, hardware encoded) could be the subject of a mini listening test. EDIT: like the lame 3.90.3 vs 3.96 test, not like the present one.

I'm not suggesting Roberto should carry out such a test - I'm just saying it would be easy to prove this one way or the other.

As you said, it probably won't happen. Let's face it, MD users aiming for decent results aren't using this setting anyway, because it adds audible artefacts so often.

Cheers,
David.

Multiformat@128kbps listening test - FINISHED

Reply #152 –
Quote
Some one could easily record all the clips to their MD recorder via a digital link (from a non-resampling sound card), and then copy them all back into a PC. Then the three versions of each clip (original, software encoded, hardware encoded) could be the subject of a mini listening test.

If have Net MD I will definitely do that test.

People who care about quality probably won't use MD.

Multiformat@128kbps listening test - FINISHED

Reply #153 –
Quote
Let's hope Apple implements VBR in their codec

The implementation in QT 6.5.1 / iTunes 4.5 is VBR see here

Multiformat@128kbps listening test - FINISHED

Reply #154 –
Quote
Quote
Let's hope Apple implements VBR in their codec

The implementation in QT 6.5.1 / iTunes 4.5 is VBR see here

ABR and recognizing silence is not the same as VBR.

If the codec knows its encoding difficult music, it still can't flex the bitrate higher.

Multiformat@128kbps listening test - FINISHED

Reply #155 –
Quote
If the codec knows its encoding difficult music, it still can't flex the bitrate higher.

How do you know its unable to do that?

Multiformat@128kbps listening test - FINISHED

Reply #156 –
Quote
Quote
If the codec knows its encoding difficult music, it still can't flex the bitrate higher.

How do you know its unable to do that?

Because you can only set bitrates and the produced file has exactly that average bitrate?

It can vary in the song, but it can't vary among songs.

Multiformat@128kbps listening test - FINISHED

Reply #157 –
Quote
The implementation in QT 6.5.1 / iTunes 4.5 is VBR

small clarification.
As far as I know there is no CBR aac encoders.
Two systems are used: VBR and ABR.
VBR - is quality based mode - bitrate is adjusted to maintain constant quality (measured by S/N ratio for example).
ABR - bitrate based mode. Average bitrate should be as defined.
CBR can (unsure) be used with aac, but by ITU specs it is not defined.
AAC always use ABR mode. If bitrate fluctuations are no more than defined by standart it is considered as CBR.
So, technically, you can consider iTunes encoding as CBR, if requrements mentioned above are met.
BTW: I remeber Ivan Dimkovic explained this somewhere on this forum, but I can't find it, period...
Quote
How do you know its unable to do that?

It seems (from testing, try it) that average bitrate remains the same...

Multiformat@128kbps listening test - FINISHED

Reply #158 –
Quote
As far as I know there is no CBR aac encoders.


iTunes 4.2 is CBR.  Frames are at a constant bitrate.  Search for Ivan's explination.  I believe he refers to it directly as CBR actually.

Multiformat@128kbps listening test - FINISHED

Reply #159 –
Quote
One more on-topic question: is it possible to send these results to portable manufacturers?

Problem is, unfortunately, codec quality is one of the least concerns of hardware manufacturers. They have much more to worry about first: hardware requirements for decoding, ease to implement on hardware, licensing fees, user demand...

Multiformat@128kbps listening test - FINISHED

Reply #160 –
Quote
iTunes 4.2 is CBR. Frames are at a constant bitrate.


Uhh. Found it at the end.
Quote
Ivan: AAC is always variable bit rate with following rules:

1. Maximum number of bits per one frame is in range from 0 to 6144, multiplied by the number of channels

See this thread: http://www.hydrogenaudio.org/forums/index....showtopic=8835&

Multiformat@128kbps listening test - FINISHED

Reply #161 –
Quote
Well, I'm not sure about MPC.
Before the test I mentioned that MPC perhaps is only as good because it uses very high bitrates on this problemsamples! But if the average bitrate is 128 for the tested qualitysetting, there should also be a lot samples with bitrates under 128kbits! Logical, isn't it?
The problem on this test is that most samples had high bitrates and the samples with small bitrates were not ranked as good!

For example you could also modify an mp3-encoder to user very high bitrates (160kbits) on difficult samples and very low (80kbits) on normal samples. In this thest it would probably be better thant the current lame-encoder but in practice there would be a lot of songs which would sound very bad!

I hope you understand what I mean. But perhaps my idea is totally wrong!?

Big_Berny

You make it sound like MPC is tweaked for winning listening tests. I believe it is tweaked to sound consistent trough the entire track. I'm no expert, but it sounds to me like you kind of confuse VBR with ABR.

I'll try to explaing my view on this matter. Then some of the experts can correct it 

The average bitrate you see with a certain VBR quality setting, is kind of a coincidence. When alot of encoded material has a certain consistent quality during the whole track, it just happens to average to e.g. 128 kbps. If you take the settings used in this test and encode metal or another demanding genre, you won't see this 128kbps average anymore. The way you explain it, it sounds like it has to sacrifice large parts of the song to be able to boost the bitrate on the hard parts. That's not the idea of VBR (more like ABR but not really that either). When a VBR setting uses only 80 kbps on certain parts, it's because it doesn't need more bits to reach the desired quality. It could have used more if it was needed.

Have you actually heard that the quality in between problem samples is lower? I'm not sure how people ended up providing the samples that they did, for this test. If they listened for problems, then from your reasoning, they wouldn't have provided the problem samples themselves, but rather the low quality parts between problem samples, as that low quality probably would stand out from the rest of the track.

CBR: Variable quality. High quality on easy parts, low quality on hard parts.
ABR: As constant quality as possible within the bitrate limitation. Use bits where they are most useful.
VBR: Constant quality.

So, from my point of view your idea is totaly wrong.  I think it would be possible, and a cynical marketing department might be tempted, but I would say that it isn't very likely the case here. These codecs are used every day, right?

IIRC Nvidia or ATI or both, tried to optimize their graphics drivers to win 3Dmark tests and make their cards look better, but I'm not sure if they sacrificed anything by doing it though.

Now, am I totally wrong? 

Multiformat@128kbps listening test - FINISHED

Reply #162 –
I wonder why the test didn't compare files that were the same size, though I know it would be a royal PITA to repeatedly encode to get that.  And then somebody would complain that time to encode should be the equalizing measure ...

Multiformat@128kbps listening test - FINISHED

Reply #163 –
Quote
You make it sound like MPC is tweaked for winning listening tests.

No! Sorry if it sounded like this! I think that this is one of the objectivest audio tests!

Sorry you misunderstood me. It's very difficult to explain it for me cause I speak german.
I'll try it again:
I only try to say that MPC perhaps only is so good in this tests because we test only samples with high bitrates! (BTW I know what ABR and VBR is!)
In this test we used quality-settings to reach an average bitrate of 128kbits, right? I know that not the average bitrate of the samples should be 128kbits, but the average bitrate of hole musiccollections with different genres. Right?
If you now look at the bitratetable you'll see that most of the samples encoded with MPC have a bitrate over 128kbits! I don't say that MPC is optimized for the listening test, but its bitrate spreading is very high! And if we only test samples with high bitrates MPC will probably give good results. But there must also be songs with a bitrate under 128kbits because of the average bitrate of approx. 128kbits! Right?
And perhaps this "easy" samples could sound bad because they have a very lower bitrate than the other codecs at this qualitysetting!

Does someone understand this theory? It's only a theory without any prove! But if you look at the testresults you can see that the sample "debussy" with a very low bitrate sounded bad with MPC!

Big_Berny

Multiformat@128kbps listening test - FINISHED

Reply #164 –
Big_Berny

you are just saying that musepack has an efficient vbr model... but saying this as if it is wrong or confusing or unfair...

this makes no sense


later

Multiformat@128kbps listening test - FINISHED

Reply #165 –
@Big_Berny:
I see what you mean after looking at the data for the Debussy sample. At first I thought your reasoning behind this low bitrate was for the wrong reason, but now I see that it probably wasn't.

I would also like to see a test like the one you suggest. If musepack or any other codec, uses too few bits in places it should have used more, then I suppose it would be useful to investigate it.

Btw: Explaining what I mean in english, isn't my strength either...

Edit: Added the part about the Debussy sample

Multiformat@128kbps listening test - FINISHED

Reply #166 –
Quote
Big_Berny

you are just saying that musepack has an efficient vbr model... but saying this as if it is wrong or confusing or unfair...

this makes no sense


later

No, that's not what I want to say. (Very happy that upNorth seems to have understood)!

I want to say that MPC has a very strong VBR-mode with very different bitrates at the same qualitysettings. In this test there were bitrates from 91kbits to 155kbits. You can call it efficient if you want.
But the problem is that: Only two of the samples we tested had a bitrate under 128kbits! And one of them was rated very bad! I just want to say that perhaps the variation of the bitrate is too high but nevertheless MPC will be rated very good in this test overall because we (almost) only tested samples with bitrates over the average (for this qualitysetting).
In the next test we should perhaps also test some samples with very low bitrates because that could be a serious problem of MPC.
If you only test difficult samples on an codec with a high bitrate variation you'll get good ratings if the codec recognizes that the sample is difficult because it will give it a high bitrate. But on the other hand it gives very little bitrate to non-problem-samples so that they perhaps have a too small bitrate. And then MPC will sound worse than the other codecs which have a smaller bitrate variation and will give a higher bitrate on this sample.


Example: Sample Kraftwerk (high bitrate)
Codecs:  iTunes  MPC    Vorbis    Lame    WMA    Atrac3
Bitrates:  128      152      135      141      127      132
Ratings:  4.30      4.78    4.30      3.32      3.11      2.29

Example: Sample Debussy (low bitrate)
Codecs:  iTunes  MPC    Vorbis    Lame    WMA    Atrac3
Bitrates:  128      98      120        108      129      132
Ratings:  4.67      3.53    4.91      3.75      3.95      4.54

You see now what I mean? Perhaps MPC is like "too efficient"!

Big_Berny

Multiformat@128kbps listening test - FINISHED

Reply #167 –
To: Big_Berny, upNorth.
Hey, guys, problem with mpc bitrate exists and was discussed on the page 4 of this thread.
See ff123 comments on this issue and how to avoid this in the future, if possible.
EDIT: grammar.

Multiformat@128kbps listening test - FINISHED

Reply #168 –
@SirGrey: I read it now, thanx.
But I think you only mentioned that it is a problem of the test and not that it could be a problem of the MPC encoder!

Big_Berny

Multiformat@128kbps listening test - FINISHED

Reply #169 –
I understand the point of Big_Berny, never thought about that.

He means that perhaps mpc could be failing on easy tracks, and as the test is featuring mostly hard tracks, it is performing good.

It might also be a matter of track loudness. As an example, the current Lame version is likely to fail in vbr when using a low volume track, as it is not estimating loudness before encoding.

Multiformat@128kbps listening test - FINISHED

Reply #170 –
Perhaps this isn't the right time for me to comment on it but I think it would have been superior to test LAME 3.96 ABR instead of VBR, i.e. "--preset 128" or something similar. At lower bitrates ABR is generally considered more effective than VBR. Theoretically VBR should be best but that's not going to be the case in the real world always.

Of course I have not been keeping track of much that is going on so perhaps I missed something.

Multiformat@128kbps listening test - FINISHED

Reply #171 –
Quote
Perhaps this isn't the right time for me to comment on it but I think it would have been superior to test LAME 3.96 ABR instead of VBR, i.e. "--preset 128" or something similar. At lower bitrates ABR is generally considered more effective than VBR. Theoretically VBR should be best but that's not going to be the case in the real world always.


This was checked before the test. Based on pre-tests, vbr was choosed.

Quote
At lower bitrates ABR is generally considered more effective than VBR

For bitrates around 128kbps, it is now time to change this consideration.

Multiformat@128kbps listening test - FINISHED

Reply #172 –
Quote
Gabriel: I understand the point of Big_Berny, never thought about that.

Me too 
ff123 mentioned that at page I point to... Never think that way before.
Interesting question to think of.
Quote
Big_Berny: But I think you only mentioned that it is a problem of the test and not that it could be a problem of the MPC encoder!

He-he.
That depends on your point of view and methodology.
Someone in that discussion porposed to use mpc with different setting for every song - to be sure avg bitrate is 128Kbit. So, in this case mpc will have no problems !   
But from another perspective, using just one setting is much more consistant...
BTW: I never used mpc for encoding, except for testing.
As I understand, it is tweaked layer 2 encoder, so for bitrates less than ~130Kbit it should produce low quality output... (?)
May be somebody familiar with mpc could explain it's behaviour ?
Quote
Gabriel: ...as it is not estimating loudness before encoding.

Oh. Thing to do for version 4 ? 

Multiformat@128kbps listening test - FINISHED

Reply #173 –
Quote
I understand the point of Big_Berny, never thought about that.

He means that perhaps mpc could be failing on easy tracks, and as the test is featuring mostly hard tracks, it is performing good.

Thank you! That's exactly what I mean! You explained most of my thoughts in one simple sentence... 

Big_Berny

Multiformat@128kbps listening test - FINISHED

Reply #174 –
Quote
To: Big_Berny, upNorth.
Hey, guys, problem with mpc bitrate exists and was discussed on the page 4 of this thread.
See ff123 comments on this issue and how to avoid this in the future, if possible.
EDIT: grammar.

I had read the whole thread already, but forgot about that discussion, sorry.

I still have the feeling that the bitrate is a topic because of the fact that this is a 128kbps test. If the motivation for adding more samples with low bitrate is only, or partly, to make the average bitrate look better (closer to 128kbps), then I would say the results of such a test would be less interesting. As ff123 has said already, some problems arise because of a too low bitrate (as seen with Debussy sample), and as I see it, that is a valid reason for using more such low bitrate samples. Add it because it is another type of problem sample, not because it makes the average closer to 128kbps.

Shouldn't the samples also be picked so that all codecs has something to struggle with? Like two samples wma struggles with, two mp3 struggles with and so on? Or maybe that would be too much to ask if as many genres as possible should be covered?

My point of view is that when it comes to bitrate, the only thing that counts is the long time average. Doing some artificial tweaking to make all codec have the same average bitrate on all these short samples, would ruin the value of the test for me. For short I agree with the way things are done already.

Then a question, or more like hearing if my understanding is right: If a codec where perfect, wouldn't it then, at a specific quality setting, recieve the same rating for all samples?

I'm sorry if all of this is old "news" and covered in another thread, then I would be greateful if someone could point me too it.
I'm just trying to see if my understanding and way of thinking is right.

Btw: As it takes me a while to write, alot has happend in the meantime. I see now that Gabriel has picked up on the point Big_Berny made.