Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Autumn 2006 Listening Test (Read 146551 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Autumn 2006 Listening Test

Reply #275 –
My choice would be to let encoders developpers provide settings when it's possible.

Autumn 2006 Listening Test

Reply #277 –
Sorry to enter the debate so late guys but this has to be said:

You all know that FHg @ 128 CBR is going to get majorly pwned right?  Personally I don't see the point in testing something that has two large disadvantages over the rest of the field.  It's going to come in last, almost for sure.*

I also think its important to remember that trying too hard to aim our tests at "Joe Average" is rather pointless.  "Joe Average" doesn't even rip his own CDs, has never heard of FHg, LAME, et al and doesn't care to.  Even if his his friend "Very Slightly Geeky Joe" makes up the majority of the CD ripping population, they are not the best target audience as, in my experience, they are happy with the results they are already getting and don't know/care if there digital audio isn't as good as the full blown "Audio Geek" that populates HA.org.

I'm not sure if anyone else looked at the freeDB statistics page brought up earlier which proves there is a significant percentage of the population using Nero to rip there CDs.  It should also be noted that this group is a far better one to target as they have for some reason decided to break with the convention, and user friendless, found in WMP and iTunes.  This group may actually be interested in our results, the "average" WMP/iTunes ripper will undoubtedly ignore/not even get a chance to see the results of this test.

As for samples I have a suite that I can upload if anyone would be interested in picking through them.

*Yes I might be proved wrong here . . . in fact I would like to be but I highly doubt that CBR@128 encoding is going to hold up very well against a field of VBR@135+.
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame

Autumn 2006 Listening Test

Reply #278 –
For the samples, sure, go ahead an post. I am more than happy to see it.

As for the FhG thingy - I already sugested Nero but the majority wanted to use WMP. Problem is that there are so many FhG implementations out there. Nearly every commercial product uses FhG, including WMP, MMJB, Nero, WinOnCD (at least the versions I tried), etc. The problem we had with CBR over VBR is that VBR does not use Joint Stereo coding (or it does, but uses Simple Stereo all the time rather than switching to M/S Stereo sometime).

Autumn 2006 Listening Test

Reply #279 –
eep!  there seems to be only 2.99mb available in the uploads section . . my suite is 43megs in wavpack -hx.

On the FhG topic I thought someone said that Nero used JS for VBR, but no time to test now I have to run out to work.
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame

Autumn 2006 Listening Test

Reply #280 –
Quote from: Sebastian Mares on
The problem we had with CBR over VBR is that VBR does not use Joint Stereo coding (or it does, but uses Simple Stereo all the time rather than switching to M/S Stereo sometime).

Alex B found an implementation with MS&SS joint stereo frames in VBR mode.

Quote from: music_man_mpc on
I'm not sure if anyone else looked at the freeDB statistics page brought up earlier which proves there is a significant percentage of the population using Nero to rip there CDs.  It should also be noted that this group is a far better one to target as they have for some reason decided to break with the convention, and user friendless, found in WMP and iTunes.

However, I'm not sure if freedb statistics provide the number of WMP users.

Autumn 2006 Listening Test

Reply #281 –
It took a while to read this thread!

I would like to re-raise the idea of Lame VBR vs ABR being included in the test. I have not seen a direct comparison of the two at 128 and it seems that this is a fundamental question that could be put to rest at last.

Autumn 2006 Listening Test

Reply #282 –
Quote from: kennedyb4 on
It took a while to read this thread!

I would like to re-raise the idea of Lame VBR vs ABR being included in the test. I have not seen a direct comparison of the two at 128 and it seems that this is a fundamental question that could be put to rest at last.


Sorry, this is not possible because it would mean having too many encoders.

I am going to download MMJB and encode some tracks.

Autumn 2006 Listening Test

Reply #283 –
Quote from: guruboolez on
It's up to you, but I wouldn't take the risk to use a personal tuning of HELIX - unless a developer gives you the green light for this.

You seem to have forgotten that the developers of Helix (specifically Real networks) offered the Helix encoder as open source with the intention that the people will experiment and why not, to improve it.
This is very easy to confirm reading very carefully the Helix thread. The Real networks team did not put objection in this, and there were discovered switches in the encoder; and even were created new ones; in other words; the Real team gave GREEN LIGHT from the beginning. In this point, that you are saying is a contradiction.

Quote from: guruboolez on
It's not that I don't trust level's commandline but there's no kind of validation for this.

That it's only and only your personal opinion based on conjectures. Even the moment, you don't have presented high bitrate tests with my 'tuned' setting that support or doesn't support your affirmation.

Autumn 2006 Listening Test

Reply #284 –
I say let's stick with Level's command line. Why not? He has tried it out and it is most likely not worse that the standard V settings. No biggie, not the end of the world if it turned out worse in the end. Nobody else has come up with any other suggestion.
//From the barren lands of the Northsmen

Autumn 2006 Listening Test

Reply #285 –
Quote from: level on
You seem to have forgotten that the developers of Helix (specifically Real networks) offered the Helix encoder as open source with the intention that the people will experiment and why not, to improve it.

Same for LAME. And it's not a reason to test the first commandline found on the web. Experience is telling us that several people are able to provide a 'tuned' commandline but most often these personal tunings are usually considered as a bad thing by developers themselves. LAME history is full of misconception and aberration.

Quote
the Real team gave GREEN LIGHT from the beginning. In this point, that you are saying is a contradiction.

Green light to what? To improve or ruin the quality? To refine or to handicap the encoder before a listening test?
The question is not about the righftulness of tuning HELIX encoder - of course it's welcome! - but to use for an '''official''' listening test a commandline which isn't validated by any publication. Your tunings are maybe excellent (and I sincerely hope it) but what we need is a bit more than a simple claim.

Quote
Quote from: guruboolez on

It's not that I don't trust level's commandline but there's no kind of validation for this.

That it's only and only your personal opinion based on conjectures. Even the moment, you don't have presented high bitrate tests with my 'tuned' setting that support or doesn't support your affirmation.

As far as I know you never provided any detail for any bitrate. And this isn't a conjecture: it's simply a fact. Testing an encoder is a very big task (and suggesting alternative commandline a big responsability). I hope that you did with 20 or 30 various samples at least and I would be very glad to see your detailed results very soon. Cheers
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

Autumn 2006 Listening Test

Reply #286 –
Quote from: DigitalDictator on
I say let's stick with Level's command line. Why not? He has tried it out and it is most likely not worse that the standard V settings. No biggie, not the end of the world if it turned out worse in the end. Nobody else has come up with any other suggestion.

OK, and we should give credit to the first person giving is personal tuning over the default one just because nobody is making an alternative suggestion? So what about LAME 3.91 -V5 -q0 -X5 --lowpass 18200 --athlower 12 --athaa-sensitivity -2 --allshort --replaygain-accurate? If nobody will bother or take time to test it and to prove me that I'm wrong shouldn't we use it?

Again, and to be clear: I'm not against Level, or Level's suggestion, or any kind of tuning. It's simply that I'm against compensating our ignorance of HELIX's encoder by using without any kind of verification a tuned commandline which maybe:
- work with few samples only
- correspond to Level's own preference
- ruin the quality in several cases.

It's a listening test and we can't act as sorcerer's apprentice.
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

Autumn 2006 Listening Test

Reply #287 –
Your deep analysis is dead on as always, Guru. I'm just starting to feel that a lot of people are losing stamina with this endless debate about encoders and settings.
//From the barren lands of the Northsmen

Autumn 2006 Listening Test

Reply #288 –
If you want know my most intimate and irrational wish on this subject it would be the inclusion of Level's tuning. But if I militate for the opposite it's because:
- we don't know if HELIX default settings are bad
- we don't know if HELIX tuned commandline suggested by Level is better, or worse
- if we test HELIX with a tuned commandline and if HELIX results aren't great, Sebastian have to take the full responsability of this failure but...
- if we test HELIX with default setting and if HELIX results aren't great AND EVEN IF A BETTER COMMANDLINE WAS AVAILABLE then the responsability would be imputable to Helix's developer's only


You can play with fire and set a test with competitors corresponding to some feelings on your own instead of sticking with the default and safer commandline. But unless your feelings were truly excellent you can't publish the final result and let your own fantasy misleadingly appear as the official representative of the tested encoder. So either HELIX will end on top or the whole test could legitimately be refuted.
One question any conducer, or tester, should keep in mind is: how could I defend my choice AFTER the results. So ask you this question. What was the basis for -V60 -X2 -HF2 -SBT450 -TX0? What would you answer in 6 or 12 months to a new member questionning this choice? We used it because someone tested it and was happy with it? It's a bit short, don't you think...
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

Autumn 2006 Listening Test

Reply #289 –
The special thing with HELIX is that the developers were asked if I understood this correctly, and they couldn't give a hint. This sounds like they don't have a very good opinion on their defaults. It sounded like that already in the HELIX thread.

From the HELIX thread I learnt that level did extensive tests though he didn't report in detail about it. AFAIK that's the only results we've got. The bigger problem to me is that level tested at a considerably higher bitrate, so we don't know whether or not his parameters are good at 130 kbps.

Anyway we're in a pretty bad situation similar to the FhG usage question.
But as we can't debate endlessly (and obviously don't get at a better decision basis) I suggest again just to use what the majority of members here think is more appropriate.
This brings responsibility quite a bit away from Sebastian and takes it to the community.

As for now I see 3 members having given their opinion: level, guruboolez and DigitalDictator.

Any more opinions?
lame3995o -Q1.7 --lowpass 17

Autumn 2006 Listening Test

Reply #290 –
Level's methodology is nicely explained here.

He apparently used 2 samples and 2 only to develop his personal commandline. His tuning is based on 2 samples (electric guitar for both). Then he used 25 samples to check the validity on the commandline. The conclusion is:

Quote
For my ears [-V120 -X2 -HF2 -SBT450 -TX0] was in ABX total transparent with all of these 25 songs of different music.

What we know is: this commandline is transparent (but at 204 kbps is it really a surprise?) but what we we don't know is if it's better than the default setting. As a consequence the most important part of the test, what should be tested first and what should interest us in this debate, is totally missing.

Moreover, Level's tunings used for -V120 basis are exactly the same than the one suggested for -V65. Is it a problem? Take a look on other encoders like LAME, Vorbis, MPC... code and check if the "inner" settings are reproduced from one VBR setting to another (like -V2 ot -V5, -quality 6 to --quality 4, ...). The answer is clearly negative. Most 'switchs' are changed, or tuned, according to the preset. How could -HF2 -SBT450 -TX0 be optimal for both 130 kbps (-V65) and 200 kbps (-120)? For me it's suspect.
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

Autumn 2006 Listening Test

Reply #291 –
I just installed MMJB (or Yahoo! Music Jukebox what it's called now) and I am afraid that I cannot use it. The High settings create files with an average bitrate of 96 kbps and a sampling rate of 32 KHz, the Ultra High mode creates files at 192 kbps with a sampling rate of 44.1 KHz. Unfortunately, there is no setting in the middle.

Autumn 2006 Listening Test

Reply #292 –
Perhaps guidelines for choosing encoder settings could be:

1. Use developers recommendations.
2. If none are given, try to find detailed tests from users and see if there is some consensus on what is good.
3. In lack of that, use the latest encoder with default settings.
"We cannot win against obsession. They care, we don't. They win."

Autumn 2006 Listening Test

Reply #293 –
Quote from: stephanV on
1. Use developers recommendations.
2. If none are given, try to find detailed tests from users and see if there is some consensus on what is good.
3. In lack of that, use the latest encoder with default settings.

Aha, then for FhG we would use mp3 surround encoder from all4mp3.com:
- no recommendations
- no comparison of various available implementations
- the encoder from all4mp3.com uses the latest FhG mp3 encoder libraries (as they claimed in the email response).

Autumn 2006 Listening Test

Reply #294 –
We can go on like this endlessly.

It's good to learn about critical issues posted here (like the concern about level's setting especially at 130 kbps), and this goes into the opinion of members taking part in this discussion.

But finally we have to choose something, and IMO that should be for any encoder that setting finding the best support here no matter whether or not we're always on solid ground.

We should keep in mind this isn't a scientific test with the ultimate meaning for the rest of the world. If we would want that we should do pre-tests or let various encoder settings take part in the test (but we're through with this I think).

Moreover I think we shouldn't try taking these things so extremely serious. Any listening test is important but can only provide for a limited experience due to the limited number of samples. Please don't understand me wrong: this doesn't degrade these tests but relativates their meaning a bit.

Let's try to get a bit quicker to the real test.
lame3995o -Q1.7 --lowpass 17

Autumn 2006 Listening Test

Reply #295 –
Choosing the latest FhG encoder (mp3sEncoder) may actually help the development if we will find regressions, bugs or problem samples. And it is also possible that sooner or later every new application will implement their latest mp3 libraries.

Also, there is growing number of mp3surround-enabled popular software:
- DivX Player 6.3.1
- Steinberg Cubase 4
- Magix Music Maker.

Edit: Nero.com site also contains "mp3 Surround audio coding technology licensed from Fraunhofer IIS, Agere Systems and Thomson."

Autumn 2006 Listening Test

Reply #296 –
I have already said my opinion about FhG. The VBR implementations look like a can of worms. MMJB VBR might be the best option if VBR is preferred. MMJB has at least some popularity and it is available for Joe Average to use. A limited pretest would be good for making sure that WMP CBR 128 does not happen to be clearly better than MMJB VBR. Perhaps Sebastian could select and make available a few (let's say five) MP3 problem samples. The active members in this thread could then post their personal ABX results. Just to make sure.

For Helix I would actually recommend using Real Player 10.5. I just tried it (what an awkward experience) and it seems to have only two 128 kbps MP3 options. CBR and VBR. Once again the 128 kbps VBR bitrates seem to be within the test target (~ about 135 kbps).

One would like to assume that Real Networks has tweaked their flagship program to produce best possible MP3 quality. This may not be practically true, but it would be acceptable and better reasoning for the used Helix setting than I have seen in this thread so far.

If Gogo is still on the list I would like to remind you of my earlier comment:
Quote from: Alex B on
I have had an impression that Gogo should be used with ABR at a bitrate like ~130 kbps. It is based on LAME 3.88 and VBR wasn't very matured at that time. Even LAME 3.9x ABR has been used at lower bitrates for example by Guruboolez until the most recent versions. I mentioned earlier that I have used command lines like this: "-a -b 192 -m j -q 2". For this test the command line could be like "-a -b 135 -m j" and the quality option 0, 1 or 2. (I have used -q 2 without any testing just because LAME 3.88 -h was mapped with the -q 2 option and it produced a good balance between the quality and speed).

Autumn 2006 Listening Test

Reply #297 –
Alex, I see no point testing an FhG implementation from a software that is several years old. I already tested the current version of MMJB and no setting comes close to 130 kbps. Using MMJB 8 or whatever is not an option.

The Helix suggestion seems reasonable to me.

What do the others think about Gogo?

Autumn 2006 Listening Test

Reply #298 –
Just a thought: the "MP3 surround command line encoder" from FHG supports CBR setting with any value (it switches between adjacent bitrates to acieve the set bitrate) so it could be used to produce 135kbps (or any other bitrate) mp3 streams.
 

Autumn 2006 Listening Test

Reply #299 –
Quote from: Sebastian Mares on
Alex, I see no point testing an FhG implementation from a software that is several years old. I already tested the current version of MMJB and no setting comes close to 130 kbps. Using MMJB 8 or whatever is not an option.

I didn't suggest to use MMJB 8.x. I assumed that the current version has not changed in this sense.

Perhaps it would be good to just use the latest WMP and the options MS has decided to make available. After all they should know what is best for their MP3 encoder.

The test idea would be then like what the big guys offer vs. LAME & Gogo at about 130 kbps. The big guys are naturally MS, Apple and Real. (I don't know how big share Real has anymore, but they were one of the first and Real Player is still very well-known.)