Skip to main content ↑ ↓
Topic: Dial-up bitrate listening test (Read 27639 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Dial-up bitrate listening test

Reply #25 –
Hello all,

I agree that 32 Kbps is an interesting data point, both for modem audio streaming, and for the audio track of a video stream at moderate broadband data rates.

A few points:
Might be interesting to hear LAME using its low bitrate tuned modes. No 32 Kbps preset

QDesign testing will need to be done with the full QDesign Music Pro 2 version, not the free one with QuickTime. However, that codec is largely deprecated for QuickTime use these days in favor of Apple's MPEG-4 AAC-LC codec.

For RealAudio, we should probably test both RealAudio 8 Stereo Music (aka "cook") and the new RealAudio 10 codec (AAC-LC based). Note the RA8 stereo codec is way better than the RA mono codec, even with mono source.

The "sweet spot" KHz for optimal quality might vary from codec to codec. I'll argue it's better to pick the right sample rate for each codec in order to let each codec show its best.


And if this goes well, maybe we should do another test at 16 Kbps. There's a lot of interest there for audio for modem rate video streams, and for mobile devices and streaming radio. The WMA Voice codec does nicely there.

I've deployed audio content down to 4 Kbps for real-world projects.
 

Dial-up bitrate listening test

Reply #26 –
Quote
For RealAudio, we should probably test both RealAudio 8 Stereo Music (aka "cook") and the new RealAudio 10 codec (AAC-LC based).
The Real AAC codec doesn´t go below 64kbps, so the only codec that can be used at this bitrate (32 or 48 kbps) is cook (ok, there was dnet (DolbyNET, AC3 with low samplerate extension) once, but that codec isn´t featured anymore for years).
But still there are four flavors that could be used:
Code: [Select]
Flavor | Codec Bitrate Label                 | Freq Resp | Sample Rate
        |                                     |           |
20     | 32 Kbps Stereo Music RA8            | 10.3 kHz  | 22050 Hz
21     | 32 Kbps Stereo Music RA8 High Resp. | 13.8 kHz  | 44100 Hz
22     | 48 Kbps Stereo Music RA8            | 13.8 kHz  | 44100 Hz
23     | 48 Kbps Stereo Music RA8 High Resp. | 16 kHz    | 44100 Hz

Dial-up bitrate listening test

Reply #27 –
Quote
AFAIK, even though the modem has a higher bandwidth than 32kbps and the protocol is usually UDP there is still some overhead bandwidth which knocks several kbps onto the stream. (Correct me if I'm wrong)

Shoutcast/Icecast use TCP. But yes, there is still overhead either way--TCP adds a bit more.

Dial-up bitrate listening test

Reply #28 –
Quote
,Jun 19 2004, 06:46 AM] For this we can test the EBU SQAM disc tracks, there are female and male speech samples: http://www.tnt.uni-hannover.de/project/mpeg/audio/sqam/

Great idea! Thanks.

Quote
Might be interesting to hear LAME using its low bitrate tuned modes. No 32 Kbps preset


Well, I guess it could be used as lower anchor...?

Quote
QDesign testing will need to be done with the full QDesign Music Pro 2 version, not the free one with QuickTime.


I have the demo of QDMC Pro 2, that lasts for 15 days. I'll use it for my test.

Quote
The "sweet spot" KHz for optimal quality might vary from codec to codec. I'll argue it's better to pick the right sample rate for each codec in order to let each codec show its best.


I agree. Some codecs already resample automatically when low bitrates are used. Other codecs might need a hand.

Dial-up bitrate listening test

Reply #29 –
Hey, Ben !!!

"Note the RA8 stereo codec is way better than the RA mono codec, even with mono source."

?????

I'm from Missouri......

Dial-up bitrate listening test

Reply #30 –
Quote
But still there are four flavors that could be used:
Code: [Select]
Flavor | Codec Bitrate Label                 | Freq Resp | Sample Rate
        |                                     |           |
20     | 32 Kbps Stereo Music RA8            | 10.3 kHz  | 22050 Hz
21     | 32 Kbps Stereo Music RA8 High Resp. | 13.8 kHz  | 44100 Hz
22     | 48 Kbps Stereo Music RA8            | 13.8 kHz  | 44100 Hz
23     | 48 Kbps Stereo Music RA8 High Resp. | 16 kHz    | 44100 Hz

Maybe we should ask Karl Lillevold if he could tell us which version to use.

Dial-up bitrate listening test

Reply #31 –
Quote
And if this goes well, maybe we should do another test at 16 Kbps. There's a lot of interest there for audio for modem rate video streams, and for mobile devices and streaming radio. The WMA Voice codec does nicely there.

That's exactly what I thought too. I hope this test will be done at 32 Kbps, 44 Khz stereo and it will be great if we could have another test at 16 Kbps.

Dial-up bitrate listening test

Reply #32 –
Quote
I hope this test will be done at 32 Kbps, 44 Khz stereo

I do not see the point in forcing codecs to use 44.1kHz.

Dial-up bitrate listening test

Reply #33 –
Are any VBR/ABR settings going to be used at this low bitrate?

If so the samples should give bitrates below 32kbps/48kbps and above 32kbps/48kbps. Ideally in the shape of a bell curve!

This way the test would not be biased towards VBR encoders. As you are testing the samples that it may have underestimated the bitrate needed.

If only CBR setting will be used then ignore this post.

Dial-up bitrate listening test

Reply #34 –
Quote
I do not see the point in forcing codecs to use 44.1kHz.

Right. I don't think anyone on his right mind would encode audio to 32kbps at 44.100Hz. I am considering either resampling everything to 22050 (I guess that's what people encoding to this bitrate would do), or letting the encoders choose the appropriate sampling rate.

Quote
Are any VBR/ABR settings going to be used at this low bitrate?


Vorbis and AAC will use VBR, unless I'm advised todo otherwise. QDesign and Real, of course, won't. I'm not sure about WMA. It seems to me it doesn't offer two-pass VBR for sampling rates other than 44.1 and 48kHz...?

Quote
If so the samples should give bitrates below 32kbps/48kbps and above 32kbps/48kbps. Ideally in the shape of a bell curve!


I agree. I'll soon start juggling samples around trying to find a nice bitrate deviation curve.

Dial-up bitrate listening test

Reply #35 –
I did a bitrate test of sorts for aoTuVb2:

First I took 18 tracks, one from each of the 18 albums I encoded for the 128 kbps multi-format test.  Each of these tracks averaged about the same as the entire album did.  Then I threw out highest, lowest, and cut from the top until I had 13 tracks which averaged 128 kbps using -q 4.35.  The actual bitrates were:

122, 123, 124, 124, 125, 127, 128, 129 , 131, 131, 132, 133, 135
This averages 128; also the median is 128, so it's pretty well centered.

Next I used these 13 tracks to encode using aoTuVb2 --resample 32000 -q -2
Bitrates were:

27, 35, 38, 36, 33, 40, 40, 41, 36, 43, 36, 46, 46 (in the same order as above)
This averages to 38; also the median is 38.

This is too high for a 32 kbps test.  At least if we assume an all music test.  I don't know what speech samples would do to the average.

ff123

BTW, 38 kbps average aoTuVb2 is actually listenable (at least for me)!

Edit:  speech samples don't help bring down the average bitrate:
female english: 40
male english: 42
female french: 40
male french: 43
female german: 39
male german: 41

Dial-up bitrate listening test

Reply #36 –
Okay, this started out as a 48kbps test and everybody complained that you can't actually stream 48kbps on most dial-up connections. So that is obviously the most important part of this test. (streamability on a dialup connection) So why use VBR for any of the samples then? VBR + streaming is not a good idea, especially if the person's buffer is not very large (usually not, people don't like to wait.) VBR on low bandwidth connections is an awful idea because when it peaks to 64 or 96kbps you may experience dropouts. Just my 2c.

Dial-up bitrate listening test

Reply #37 –
Quote
Okay, this started out as a 48kbps test and everybody complained that you can't actually stream 48kbps on most dial-up connections. So that is obviously the most important part of this test. (streamability on a dialup connection) So why use VBR for any of the samples then? VBR + streaming is not a good idea, especially if the person's buffer is not very large (usually not, people don't like to wait.) VBR on low bandwidth connections is an awful idea because when it peaks to 64 or 96kbps you may experience dropouts. Just my 2c.

Just exploring the bitrates and the test assumptions.  Assume that there was a vorbis vbr setting that averaged 32 kbps.  Would the deviation about that bitrate be small enough to allow it to be considered in a dialup test?  If you subtract 6 kbps from each of the bitrates I found, you'd get 40 kbps for the highest track.  Seems like it's at least arguable that it could work.

In any case, another setting needs to be found.  Bitrate managed may be the only option.

ff123

Dial-up bitrate listening test

Reply #38 –
Ok, here's aoTuVb2 --resample 22050 -b 32

where -b is bitrate managed mode (but still VBR).  The tracks now are:
29, 31, 34, 34, 33, 34, 36, 36, 33, 37, 34, 35, 34, for an average of 34 kbps.

This is better, but now we're in the debatable range of whether or not a track which averages 37 kbps is acceptable or not.  BTW, I could not use -b 32 if I resampled at 32 kHz.

ff123

Edit:  these sound a lot worse than the -q -2 files

Edit2:  using -b30 reduces bitrates another 2 kbps, to what is most likely acceptable on even the high tracks.

Are Ayumi's changes even operating in this mode?  Why is 22 kHz resampling forced at this bitrate?  Maybe 32 kHz would sound better.

Dial-up bitrate listening test

Reply #39 –
Not sure I'm upto date on all streaming codecs, but some things to consider:

1) Why not hard limit upper range of bit rate?

I think that would be a relatively fair assumption for streaming conditions. Using VBR and then calculating arbitrary averages isn't necessarily going to work under real streaming conditions. If average path is chosen, then it'll be more like a "pseudo streaming" codec test, no?

So, would CBR or hard-limited max bit rate be a better choice, considering this is a streaming test and that the results should reflect that (and not pseudo-idealized conditions)?

2) I also vouch for using at least two langauges (in speech samples) if possible.

This is not about trying to represent all the world languages, but trying to find for example languages that have widely differing fricative frequency/type and intonation. These can throw different problems for different codecs. I'd suggest Engl/Ger as minimum two myself. Spanish is much lower in frequency on certain type of hard fricatives as is French.

Excellent test idea, btw! Thanks again to rjamorim for being brave by facing the task (again!).

Dial-up bitrate listening test

Reply #40 –
Quote
Not sure I'm upto date on all streaming codecs, but some things to consider:

1) Why not hard limit upper range of bit rate?

This can be done with vorbis at 22.05 kHz resampling down to a hard upper limit of 35 kbps.

In other words:  --resample 22050 -M 35

In order to hard upper limit at 32 kbps, though, resampling has to be done at 16 kHz.

I think it is probably reasonable to set the hard upper limit at 35 kbps.  I think most tracks will probably average (as opposed to hard limit) at 32 kbps.  I can try the 13 tracks later.

ff123

Dial-up bitrate listening test

Reply #41 –
Quote
Are Ayumi's changes even operating in this mode?  Why is 22 kHz resampling forced at this bitrate?  Maybe 32 kHz would sound better.

I think aoTuV shouldn't be used with any bitrate management options like -b, -m, nor -M. At the present time, he doesn't seem to tune his encoder for using with these options.

Dial-up bitrate listening test

Reply #42 –
Yes, please remember that at bitrates this low even a few kbps deviation from 32kbps (CBR/ABR, or average if we decide to use VBR) will make a large difference to the final output, and file size/bitrate IS very important here, as opposed to when testing files which are targeting a quality, not a bitrate.

Dial-up bitrate listening test

Reply #43 –
Bitrates for 13 tracks using aoTuVb2 --resample 22050 -M 35

27, 29, 33, 32, 31, 32, 34, 33, 31, 34, 32, 33, 32

average is 32
Sound is fluttery, like the official encoder; can probably use either version for the test.

ff123

Dial-up bitrate listening test

Reply #44 –
In a low frequency domain, aoTuV beta2 will bring the same result as 1.0.1. Therefore, for the moment, there is no meaning which turns down 1.0.1 and chooses aoTuV..

Dial-up bitrate listening test

Reply #45 –
Quote
In a low frequency domain, aoTuV beta2 will bring the same result as 1.0.1. Therefore, for the moment, there is no meaning which turns down 1.0.1 and chooses aoTuV..

Too bad, because that extra 6 kbps (on average), the resampling at 32 kHz, and your extra quality enhancements makes a big difference in sound quality.

ff123

Dial-up bitrate listening test

Reply #46 –
Quote
Dunno the case with speex, but most voice coders fail VERY badly on music and complex sound effects.

Spees does fail "VERY badly" on music, only "badly" ;-) Seriously, at CBR ~34 kbps, it's possible to have recognizable music with Speex, but the quality is definitely not on par with what you'd get for Vorbis. Probably not worth including in this test.

Quote
I think Speex belongs to a test dedicated to vocodecs. I already started discussing such test with jmvalin, hopefully he'll have time to conduce it (or someone else might become interested).

Don't have much time to organize it, but I can provide tips on what codecs/settings to use.

Dial-up bitrate listening test

Reply #47 –
I know there are already six encoders being used in this test, and even though this has been the limit in the past would it be possible to include musepack using the telephone preset (quality 2) in this test? It may not be as well supported as the others, but musepack has done suprisingly well in the 64kbps test, maybe it can do well at 32kbps too.

Dial-up bitrate listening test

Reply #48 –
Quote
but musepack has done suprisingly well in the 64kbps test

Musepack was not tested in my 64kbps test


@ff123: Thank-you very much for verifying the behaviour of vorbis encodes. That's much appreciated.

Dial-up bitrate listening test

Reply #49 –
Quote
... but musepack has done suprisingly well in the 64kbps test ...

On 128 kbps multiformat listening test, but not at 64 kbps (not tested: useless, crap quality).
musepack at 32 kbps might compete... as anchor.

EDIT: Roberto was faster.
 
SimplePortal 1.0.0 RC1 © 2008-2019