Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: 128kbps Extension Test - FINISHED (Read 73949 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

128kbps Extension Test - FINISHED

Reply #125 –
@ff123: thanks for the kind words. (And thanks to everyone else that praised the test. That is _really_ welcome)

128kbps Extension Test - FINISHED

Reply #126 –
Edit: post split in two parts due to @#$%& forum bugs.

Quote
Vorbis is a product of the free open source concept rather than intensive and sophicated research and development by the big companies like Fraunhofer, Dolby, or Microsoft.


You seem highly biased toward Vorbis. If you're looking for a product of the free open source concept, why not Musepack? The next version is open source, and it won the test.

Quote
I thought we should do this test to see how different codeks can handel common problem samples at same average bittrate.


They ARE handling it at the same average bitrate. It might not be the same ABR for the samples, but it is for the overall album or collection. This test is great because it's not just testing a codec's quality, but its ability to exploit the advantages of VBR over CBR as well.

Quote
I think almost everyone here who encodes music him self with lossy codek will not use 128kbps average size without any special reason.


Depends on what you call special reason. 128kbps is a very popular choice for broadband streaming, and some of the samples proved it might be a good fit for some portables as well.
Happiness - The agreeable sensation of contemplating the misery of others.

128kbps Extension Test - FINISHED

Reply #127 –
Quote
To test efficiency, which i believe is what should be the goal for a 128kps listening test, the bps should be more similar.


Efficiency? How do you define efficiency? Speed? Quality? Optimum mixture of the two? It doesn't matter what the answer is, because Roberto's test clearly states on the presentation page "Its purpose is to find what encoder outputs the best quality on bitrates around 128kbps".

Quote
I only would like to see how much closer it would be if we fine tuned it in the future, that's all.


If the test's purpose doesn't change, I don't see how your proposed changes would improve the method.

,
Radu
Happiness - The agreeable sensation of contemplating the misery of others.

128kbps Extension Test - FINISHED

Reply #128 –
Quote
Edit: post split in two parts due to @#$%& forum bugs.

Quote
Vorbis is a product of the free open source concept rather than intensive and sophicated research and development by the big companies like Fraunhofer, Dolby, or Microsoft.


You seem highly biased toward Vorbis. If you're looking for a product of the free open source concept, why not Musepack? The next version is open source, and it won the test.

Biased?  There is nothing biased about stating facts.  The fact is Vorbis source code has been open for quite a while yet I haven't seen musepack source code in the public domain that's for all to see and contribute for as long.  Whether it is open source in the next version is something in the future, not the past, thus my statement is accurate.

Anyway I sense this bias against those who make favourable comments about Vorbis.  Is it really fundamentally wrong to acknowledge the work put into Vorbis that sees it outperform even the open-source and highly tuned lame mp3 encoder?  Is it fundamentally wrong to acknowledge the potential of Vorbis to improve as we've seen in Garf's tunings?

Or maybe we should just throw away every coder in development and use mpc instead?  I believe competition is the way to progress and we've all acknowledged (including myself) the greatness of mpc, as demonstrated in this test.  mpc has defined the state-of-art and provides the benchmark for other competing coders to emulate and attempt to surpass.

128kbps Extension Test - FINISHED

Reply #129 –
Quote
Quote
I thought we should do this test to see how different codeks can handel common problem samples at same average bittrate.

They ARE handling it at the same average bitrate. It might not be the same ABR for the samples, but it is for the overall album or collection. This test is great because it's not just testing a codec's quality, but its ability to exploit the advantages of VBR over CBR as well.

Is there a big differece if we encode something with ABR to 128kbps and encode with VBR that same sample in to same size? Should'nt that VBR still be at least as good as ABR encoded sample at least if encoder is same?

Quote
Quote
I think almost everyone here who encodes music him self with lossy codek will not use 128kbps average size without any special reason.

Depends on what you call special reason. 128kbps is a very popular choice for broadband streaming, and some of the samples proved it might be a good fit for some portables as well.

But does ABR or VBR work well in streaming? I was under that impression that CBR is only choise for streaming? And i guess that we were'nt testing samples for streaming.
And i ment with that special reason for situation where size matters and quality isn't so high priority.


rjamorim: I'm sorry roberto, but i had to extract my frustrated opinnions from this subject. I mean no harm to you. Overall this test was werry well made and handelled.

128kbps Extension Test - FINISHED

Reply #130 –
@rjamorim

Thank you again for the test! I'd appreciate tests like that in the future!

Kind regards, fileman.

128kbps Extension Test - FINISHED

Reply #131 –
Quote
But does ABR or VBR work well in streaming?

With buffering (which most steaming audio progs use), and a small safety margin for bandwidth, ABR should work for streaming (VBR would be iffy), assuming the range used isn't TOO big.

Uses for 128kbps are actually pretty common.  Lots of people have flash-based mp3 players with limited space, and 128kbps VBR is the right tradeoff level for a lot of people.  These sorts of players are tending towards getting embedded in everything (cell phones, PDAs, probably watches before too long).

Also, right now, almost all the on-line music retailers (Apple's iTunes, eMusic, MusicNow, etc.) use 128kbps.  Their claim that it's CD quality is obviously a joke (I'd call it fraud), but that doesn't prevent them from using that bitrate.  I know everybody here would like mpc --standard or some such, but until that happens (never!) people who want to buy music online are stuck with 128kbps.  I'd imagine they'd be interested in what performs the best at that bitrate.
I am *expanding!*  It is so much *squishy* to *smell* you!  *Campers* are the best!  I have *anticipation* and then what?  Better parties in *the middle* for sure.
http://www.phong.org/

128kbps Extension Test - FINISHED

Reply #132 –
I was thinking about the variability in bitrate, and realised that the people questioning the use of bitrates for the sample other than 128kbps aren't going far enough.  What we really need to do is cut each sample into 1 second intervals, and make sure that each of those is exactly 128kbs.  That should remove the variability.

No, wait, slice each one into 0.1 second intervals, and make sure that each is exactly 12.8kbs.

no, wait, slice each one into 0.01 second intervals.....

Cheers, Paul

(edit:spelling)

128kbps Extension Test - FINISHED

Reply #133 –
Quote
Is there a big differece if we encode something with ABR to 128kbps and encode with VBR that same sample in to same size? Should'nt that VBR still be at least as good as ABR encoded sample at least if encoder is same?

First: How do you expect me to encode something to ABR in Musepack?
If you didn't notice yet, WMA Pro, AAC and Lame are already in ABR mode.

Second: Let's take Vorbis, that is the only format in the test with ABR and an usable VBR mode: How can you be sure that the ABR mode will be at least as good as VBR? Can you imagine the flood of complaints from vorbis zealots if I use anything but the best setting?

Again (hopefully for the last time): This test is about quality, not bitrate, encoding speed, decoding speed, encoder price.. (when commercial). These other factors are of no interest to the final rating. Therefore, I chose the best quality setting for each format that outputs 128kbps on a large average (the quality setting test thread).

128kbps Extension Test - FINISHED

Reply #134 –
Quote
... I chose the best quality setting for each format that outputs 128kbps on a large average (the quality setting test thread).

The truth, only the truth and nothing but the truth...

I fail to see why it is so hard to understand for some people...

I agree 100% (or more  ) with the methodology, the settings chosen, the procedure to choose them, and everything in the test.

Great test, very good work, and my most honest congratulations, hoping they will help to get you encouraged to go on performing this sort of tests.

People: read the thread in which the quality settings were chosen! All of it! You should be use to reading long threads by now if you are in HA!

128kbps Extension Test - FINISHED

Reply #135 –
How you've not responded to the 1000th complaint without resorting to all out apocalyptic warfare I don't know, you are a man of infinite patience Roberto

I haven't really been involved with the test at all, yet I've been pissed off at people for not taking time out to read how the test was designed.. . aaaagh!

Go RJAmorim!
< w o g o n e . c o m / l o l >

128kbps Extension Test - FINISHED

Reply #136 –
Quote
I haven't really been involved with the test at all, yet I've been pissed off at people for not taking time out to read how the test was designed.. . aaaagh!

Not to mention that these people kept silent when Roberto welcomed criticism and advice BEFORE starting the test. doh!

128kbps Extension Test - FINISHED

Reply #137 –
Perhaps in the next test, all the test samples should be 128kbps more or less exactly, by way of adjusting -q levels for the VBR codecs. More work for roberto, no doubt, but I'm sure this would be an arrangment which more people would be happy with in terms of 'fairness'.

My conclusion for this test would simply be that this test confirms that significant improvements have been made in audio compression since MP3, despite those people claiming that "MP3 should be good enough, we should all continue using it and ignore all those incompatible new formats". MPC, Vorbis, WMAPro and AAC have shown to be significantly better than MP3. Between themselves, there are none which are significantly better than the other, so your choice will probably depend on whether you are an OSS advocate, M$-lover, MPC-diehard or Mpeg standards advocate. Or something like that.

128kbps Extension Test - FINISHED

Reply #138 –
Quote
Perhaps in the next test, all the test samples should be 128kbps more or less exactly, by way of adjusting -q levels for the VBR codecs. More work for roberto, no doubt, but I'm sure this would be an arrangment which more people would be happy with in terms of 'fairness'.

This way is more fair, I for one WOULD complain if the test was done the way you purpose.
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame

128kbps Extension Test - FINISHED

Reply #139 –
Quote
More work for roberto, no doubt, but I'm sure this would be an arrangment which more people would be happy with in terms of 'fairness'.

"More people" ? No, this will satisfy people who doesn't understand VBR principle in a practical point of vue, and made angree (and probably boycott) all the others.

Other thing to do : use non-difficult sample, in order to avoid VBR encoder's bitrate inflation. Then, mpc, vorbis... will probably reach their statistical average bitrate, without any tweaking. But I'm not sure that many pepole will be able to ABX one of this encoding. As you said, and I'm agree with you, new generation of encoder are really better than the best mp3 one.
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

128kbps Extension Test - FINISHED

Reply #140 –
Roberto, I would just like to say that this was an excellently run test and I consider its results very credible, great work!  I look forward to participating in the next test you put togther (tentative planning it to be all mp3, are you not?).  The only thing I would have liked to see in the results would be some sort of transparency data, like "x-number of users found this sample with this encoder totally transparent".

regards,

-Tyler
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame

128kbps Extension Test - FINISHED

Reply #141 –
Quote
I look forward to participating in the next test you put togther (tentative planning it to be all mp3, are you not?).

Thanks

Actually, the MP3 codecs test is on low priority now. I already started planning it (I'm thinking of comparing Lame, MP3pro, MP3enc, FhG fastenc and Radium, with Blade as anchor again), but it's really too early to discuss that now.

The next test will be at 64kbps. I plan to compare Vorbis, HE AAC, MP3pro, WMA pro and Real Audio Cook, using 7kHz and 3.5kHz lowpass as anchors. If all goes as planned, I'll start a calling for samples next week and the test starts on September 3rd (with a pre-test thread starting a week before. And this time I expect everyone to participate :-P)

After that, I plan to test several vocoders, with the help of JMValin. The preliminary competitors would be Speex, MPEG4 CELP, Acelp.net (As implemented in Windows Media encoder), GSM and g729. I don't know if such test needs an anchor, if not, I might throw in another encoder, like VoxWare or Qualcomm PureVoice.

The vocodec test will probably be separated in two perts, wideband and narrowband.

Regards;

Me.

128kbps Extension Test - FINISHED

Reply #142 –
@Guruboolez: Fortunately, for the next test, I'll be able to use non-difficult samples, since it's at 64kbps. The only "problem" sample I would keep is Waiting, since that's one of the most interesting samples I've seen - it doesn't look at all like a problem sample, but encoders have a hard time with it (check the aac test, for example)

Regards;

Roberto.

128kbps Extension Test - FINISHED

Reply #143 –
At some point, we probably need to split off for pre 64-kbit/s topics.

Regarding ABC/Hr, it probably needs to be recompiled to change the scale descriptions to better suit a lower bitrate test:  i.e., "excellent, good, fair, poor, etc", instead of "imperceptible, perceptible but not annoying, etc"

I'm working right now to add an obscuring feature to hide the contents of the config and results files, but I don't know if that will be ready by the time I take off for a 3 week vacation.

ff123

128kbps Extension Test - FINISHED

Reply #144 –
Quote
After that, I plan to test several vocoders, with the help of JMValin. The preliminary competitors would be Speex, MPEG4 CELP, Acelp.net (As implemented in Windows Media encoder), GSM and g729. I don't know if such test needs an anchor, if not, I might throw in another encoder, like VoxWare or Qualcomm PureVoice.

This sounds cool. I have'nt try any speech codek before, exept almost daily use of GSM...

128kbps Extension Test - FINISHED

Reply #146 –
Interesting.

Regarding WMA9Pro "1.0": In near future there will be an update to WM9, but it is unclear when (there's a possibility MS might ditch the plan and wait for Longhorn) and if it will just affect the player or also the codecs.

--edit: typos and grammer

128kbps Extension Test - FINISHED

Reply #147 –
Quote
The next test will be at 64kbps. I plan to compare Vorbis, HE AAC, MP3pro, WMA pro and Real Audio Cook

i am really interested in a 64kbps test but wouldnt it be better to wait a little bit more as
- nero he-aac is new and probably needs more tuning (next release on 31 august)
- monty already plans to tune vorbis for low bitrates (btw the next official vorbis release will be on 1st september -> 1.0.1 only bugfixes)

and i think you meant normal wma9 as the pro codec cant be used below 128 afaik
I know, that I know nothing (Socrates)

128kbps Extension Test - FINISHED

Reply #148 –
Quote
i am really interested in a 64kbps test but wouldnt it be better to wait a little bit more as
- nero he-aac is new and probably needs more tuning (next release on 31 august)
- monty already plans to tune vorbis for low bitrates (btw the next official vorbis release will be on 1st september -> 1.0.1 only bugfixes)

Well, the test will start after Nero's next release (it's planed for Sept 3rd), so I think it will be available anyway.

And how long will I have to wait until Vorbis 1.1 is released? If it follows Xiph's release schedule, this test will start around October 6th, 2004 :B
(It took more than an year for a minon-minor new version (1.0.1), how long will it take for a minor version (1.1)?)

Do you have any info from Monty if he at least started working on tuning 1.1 already? If he didn't, I see no point waiting months for only one codec.

Quote
and i think you meant normal wma9 as the pro codec cant be used below 128 afaik


Oh, yeah? I sincerely don't know.

Can someone with Windows Media Encoder 9 check out if you can get a 64kbps two pass VBR encode out of it in WMA pro?

I could chack it, but it's already 5:15AM here  :-/

Thanks for the info, Bond.

Regards;

Roberto.

128kbps Extension Test - FINISHED

Reply #149 –
OMG! I must stop that! I want to take a break, not start talking about the 64kbps test already.

people: Nevermind the request about Windows Media. I'll have time to verify that later, there are still almost 4 weeks until the test starts.

Please, let's not discuss the 64kbps test here.

The test will start to be discussed about 10 days before the test start.

Thanks for your comprehension.

R.