Multiformat 48 kbps Listening Test

Topic: Multiformat 48 kbps Listening Test (Read 86847 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

Multiformat 48 kbps Listening Test

Reply #25 – 2006-11-02 00:36:59

Quote from: Junon on 2006-11-02 00:06:20

Alternatively, since 16 bits files are 2/3 the size of 24 bits ones, you could also allow the WMA Pro codec to participate with 48 kbps*1.5=72 kbps.

It's wrong, because lossy encoders aren't working with fixed bitdepth. And even if you were right, it would be the developer decision to "resample" to 24 bit all 16 bit source material and this behaviour should therefore be tested as it - according to the developers' original decision.

Multiformat 48 kbps Listening Test

Reply #26 – 2006-11-02 00:50:08

Quote from: guruboolez on 2006-11-02 00:36:59

It's wrong, because lossy encoders aren't working with fixed bitdepth.

Ah, I see, thanks for the clarification. I didn't know about that, therefore I relied on calculations based on uncompressed materials' behaviour.

Multiformat 48 kbps Listening Test

Reply #27 – 2006-11-02 00:55:09

Quote from: Junon on 2006-11-02 00:50:08

Quote from: guruboolez on 2006-11-02 00:36:59
It's wrong, because lossy encoders aren't working with fixed bitdepth.

Ah, I see, thanks for the clarification. I didn't know about that, therefore I relied on calculations based on uncompressed materials' behaviour.

Interestingly, you can ask the Windows Media Format SDK decoder to give you 24bit samples for WMA standard, but the least significant 8 bits are always 0. I suspect that Microsoft's decoder implementation is integer based (it's either that or it's stupidly doing a 32bit float->16bit PCM ->24bit PCM conversion)

Multiformat 48 kbps Listening Test

Reply #28 – 2006-11-02 00:59:41

Anyway, WMAPro at CBR 96 kbps and less is necessary "16 bit" ("24 bit" doesn't seem to be available for those bitrates - not with graphical frontends as Winamp & VUplayer at least).

Multiformat 48 kbps Listening Test

Reply #29 – 2006-11-02 14:37:21

Hmm, this is really bad news. I know that 2-pass VBR results would not mimic real world usage, but wouldn't this still be better than testing plain CBR? I know it was used in some of Roberto's tests.

Multiformat 48 kbps Listening Test

Reply #30 – 2006-11-02 15:02:16

foobar2000 doesn't compute correctly the average bitrate for WMA VBR files (tested with WMA STD q10 and with foobar2000 0.9.4.1).

Example with a long file (~20 minutes).

TECHNICAL INFORMATION

Code: [Select]

File Name : [STD q10] E01_MODERN_CHAMBER_A_brass.wma
File Path : C:\music\[STD q10] E01_MODERN_CHAMBER_A_brass.wma
Subsong Index : 0
File Size : 4 409 475 bytes
Last Modified : 2006-11-02 02:27:14
Duration : 18:50.327 (49847421 samples)
Sample Rate : 44100 Hz
Channels : 2
Bitrate : 56 kbps
Codec : WMA
Codec Profile : WMA V2
Encoding : lossy

The file size and the length are good, but the computed bitrate is necessary false:

Code: [Select]

1130 secondes
4 409 475 bytes x 8 = 35275800 bits
35275800 bits for 1130 seconds => 31217 bits per seconds
                               => 31,2 kbps [1kb=1000b]
                               => 30,5 kbps [1kb=1024b]

4.409.475 bytes for 18 minutes and 50 seconds correspond to 30...31 kbps and not 56 kbps (which is what foobar2000 reports for this file).

Now lets see what other softwares are saying:

• Windows Media Player 11 => 30 kbps CORRECT
• Winamp 5.31 => 30 kbps CORRECT
• MrQuestionMan 0.8 => 44 kbps WRONG

Winamp brings additionnal information in the file property box. I noticed and reported in the past that three fields in metadata are dedicated to the bitrate: "Bitrate", "CurrentBitrate" and "OptimalBitrate". And the values are:

Code: [Select]

Bitrate        = 55917
CurrentBitrate = 30720
OptimalBitrate = 30720

I guess that foobar2000 reports the WMA "Bitrate" field: 56 kbps for "55917". But despite its name it's the wrong one. foobar2000 shouldn't therefore be used as bitrate measurement for WMA VBR files (CBR must be OK). The greatest discrepency I found was 80% (32 kbps for one file while foobar2000 says 58 kbps) but in most cases it's much lower (~10% on average).

To finish, the average bitrate for my set of 150 classical COMPLETE tracks (~16 hours) was 41.43 kbps. It's too low. But I noticed in the past that WMA uses more bitrate for other musical genres; I'm quite certain that 48 kbps would be reached with alternate bitrate table.

Multiformat 48 kbps Listening Test

Reply #31 – 2006-11-02 15:29:05

This is WMA Standard 9.2, quality based VBR with Q10, correct?

Multiformat 48 kbps Listening Test

Reply #32 – 2006-11-02 15:34:56

Yes (sorry, it was clear for myself only):
WMA 9.2 ['Standard'] VBR 10 - encoded with Winamp from flac source.

Multiformat 48 kbps Listening Test

Reply #33 – 2006-11-02 15:39:11

Quote from: guruboolez on 2006-11-02 15:02:16

foobar2000 doesn't compute correctly the average bitrate for WMA VBR files

I agree with you, maybe CBR too.

Multiformat 48 kbps Listening Test

Reply #34 – 2006-11-02 15:40:04

OK, what to do with WMA Professional 10 then? The two possible solutions for me are: encode to CBR or encode samples to 2-pass VBR as it was done in previous listening tests.

Quote from: Jillian on 2006-11-02 15:39:11

Quote from: guruboolez on 2006-11-02 15:02:16

foobar2000 doesn't compute correctly the average bitrate for WMA VBR files

I agree with you, maybe CBR too.

CBR is CBR. There isn't much you can do wrong computing CBR bitrate.

Going to install Ubuntu 6.10...

Multiformat 48 kbps Listening Test

Reply #35 – 2006-11-02 16:00:10

Quote from: Sebastian Mares on 2006-11-02 15:40:04

OK, what to do with WMA Professional 10 then? The two possible solutions for me are: encode to CBR or encode samples to 2-pass VBR as it was done in previous listening tests.

• The past use of 2-pass encoding on short samples was a mistake. Now we're aware that 2pass can't be used like that, we can't reproduce this mistake anymore.

• Well, if VBR isn't possible with WMAPro we should either go for CBR or give up with WMAPro for this test... The latter is not the best solution IMO.

I would:
- determine what exact bitrate WMA STD correspond to (is it 48? 50? 51?)
- adapt HE-AAC and Vorbis to match this bitrate (it should be possible if Nero Digital will be used)
=> three competitors will have highly comparable settings
- then for WMAPro, which would be slightly handicaped, I would introduce in the final representation of results a kind of separation (a border, a different color...) to recall that there's a slight limitation to any direct comparison of results between WMAPro and the three other competitors.

Multiformat 48 kbps Listening Test

Reply #36 – 2006-11-02 16:11:40

Quote from: guruboolez on 2006-11-02 16:00:10

- adapt HE-AAC and Vorbis to match this bitrate (it should be possible if Nero Digital will be used)

It's worth noting that I can supply a custom CT-based Winamp encoder to hit any bitrate you'd like.

Multiformat 48 kbps Listening Test

Reply #37 – 2006-11-02 16:27:12

It would be nice but I wonder if we shouldn't follow the results of the previous listening test and use Nero Digital instead of Coding Technology (CBR only) implementation.

Multiformat 48 kbps Listening Test

Reply #38 – 2006-11-02 16:48:45

Quote from: guruboolez on 2006-11-02 16:27:12

It would be nice but I wonder if we shouldn't follow the results of the previous listening test and use Nero Digital instead of Coding Technology (CBR only) implementation.

small correction. CT is "short window ABR". I presume CT's goal is streamability, so it doesn't stray away from the designated bitrate too far or for too long, but it's definitely not CBR.
example bitrate distribution (on a 51kbps file I just encoded as a test): http://filebox.vt.edu/users/ballison/51.png

Multiformat 48 kbps Listening Test

Reply #39 – 2006-11-02 20:09:30

ATRAC3+ uses CBR only, right?

Real seems to use 44 kbps CBR which is not an option for this test.

Multiformat 48 kbps Listening Test

Reply #40 – 2006-11-02 20:27:50

Quote from: Sebastian Mares on 2006-11-02 20:09:30

ATRAC3+ uses CBR only, right?

Yeh I can conferm that, I've been useing it for a year or so!

CBR only!

Multiformat 48 kbps Listening Test

Reply #41 – 2006-11-03 08:26:24

If WMA 10 is going to be used with CBR, I would like to add ATRAC3+ there. Some players, especially in Asia IIRC, support the format so it would fit into the portability category.

Multiformat 48 kbps Listening Test

Reply #42 – 2006-11-03 08:48:55

I don't like the preoccupation with a nominal and, most likely, inaccurate bit rate measurement -- what matters is the size of the file. My point is that the files being compared should all be as close in size as possible.
Surely we all know that HE-AAC is the codec for these rates, so both Nero and CT should be included. Also, wasn't there supposed to be a new Nero encoder before year end? It would be silly to do the test just before Ivan Dimkovic shows us new tricks...

Multiformat 48 kbps Listening Test

Reply #43 – 2006-11-03 09:03:08

Most likely, there won't be any significant change in the 48 Kbit/s HE-AAC codec, so I would definitely propose to use the codec used in CT vs. Nero shootout.

Question is, what bit-rate mode to use - as Nero encoder supports several bit rate allocation methods (ISO CBR, more relaxed ABR, VBR 1 and 2 pass)

I would personally go for ABR, as this was the mode used to test against Coding Tech.

Multiformat 48 kbps Listening Test

Reply #44 – 2006-11-03 09:45:32

Quote from: Sebastian Mares on 2006-11-02 20:09:30

Real seems to use 44 kbps CBR which is not an option for this test.

Isn't Real just some flavour of AAC in a Real container nowadays?

Multiformat 48 kbps Listening Test

Reply #45 – 2006-11-03 10:31:52

I think that "aoTuV" should be used with a quality base.
That is because it is tuned up in quality mode.
I have done only the minimum test about managed mode.

Multiformat 48 kbps Listening Test

Reply #46 – 2006-11-03 11:23:36

Thanks for the feedback Aoyumi and Ivan. I will use the settings you guys recommend since you know the internals of your codecs better than anyone else.

Edit: Do you guys think 6 contenders are too much? If not, my list of contenders is:

WMA Standard 9.2
WMA Professional 10
Nero Digital HE-AAC
CT HE-AAC (Winamp)
Ogg Vorbis AuToV Beta 5
ATRAC3+

If 6 is too much, we could exclude either CT HE-AAC (Winamp) or ATRAC3+.

What to use as anchors? Do we need both low and high anchor? If low anchor is needed, I would go with LC-AAC at 48 kbps. If high anchor is needed, I would go with 80 or 96 kbps LC-AAC.