Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: MP3 Listening Test at 128 kbps (Read 209347 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MP3 Listening Test at 128 kbps

Reply #100 –
RealPLayer and the command line Helix encoder from Rarewares seem to use quite different VBR settings by default. Helix @ -V60 was 3.6 x faster than RealPlayer in my test.

I have run my bitrate and speed tests and just started to prepare my report. It may take an hour or so. I encoded my usual "25 complete tracks" test set 12 times using the discussed encoders and some possible setting options.

MP3 Listening Test at 128 kbps

Reply #101 –
Another "speed" consideration is that encoding potential far outstrips ripping potential, and the disparity will only grow. Looking at Tom's Hardware CPU chart, it looks like just about any new machine purchased in the past three years or so is capable of encoding an entire CD with LAME in five minutes or less, though I don't remember which settings they use.

MP3 Listening Test at 128 kbps

Reply #102 –
*Lame settings: for 3.98, I'd suggest "-V5" (vbr-new is the default vbr mode in 3.98), but I'm not sure if it would not be higher than 128kbps.

*High anchor: I agree that it could be dropped, as we are expecting some already high results

*Gogo: is anyone still using it?

*"Radium" crack: I think that the results would be very similar to Audioactive Production Studio

*Lame fast setting: well, I don't think that there is a need for any test to know that current Lame would produce quite lower quality than FhG if we want to compete with speed in mind.

MP3 Listening Test at 128 kbps

Reply #103 –
Here are my results. For iTunes and RealPlayer I used the standard GUI programs. For the command line encoders I used Speeks' Batchenc frontend. The WMP11 mp3 codec was accessed by Nyaochi's Acmenc and Speek's Batchenc. The test bench was a 2.8 GHz P4. Before the speed tests I tried to disable all unnecessary background processes. The source files were in wave format on one HD and the target HD was separate. I measured the encoding times with a digital stopwatch. I didn't run several passes that would have been needed for lowering the error margin, but I think the results are accurate enough for this purpose. For measuring the bitrates I used Encspot Pro's "Complete Scan" feature (some other programs may not show correct bitrates with files that don't contain a Xing header).

The test file set consists of 25 selected complete tracks of various genres. The total duration is one hour, 31 minutes and 34 seconds.

The encoding speeds and average bitrates:


The individual bitrates of the VBR files:


A chart of the VBR bitrates:


Some readers may also find the following table interesting (Album = the 25-track test set):



I uploaded the test data here (Excel file & Encspot Pro reports):
http://www.hydrogenaudio.org/forums/index....st&p=515301

MP3 Listening Test at 128 kbps

Reply #104 –
May I suggest that LAME ABR 128 be included in the test, aside from -V5 which is not a true 128 setting?
"Listen to me...
Never take unsolicited advice..."

MP3 Listening Test at 128 kbps

Reply #105 –
Thanks a lot for your test Alex. I am wondering if the highest iTunes setting (VBR quality 7) would increase bitrate too much. Have to do some testing myself.

Quote from: Rio on
May I suggest that LAME ABR 128 be included in the test, aside from -V5 which is not a true 128 setting?


No.

By the way, seeing that FhG 4.1 VBR is only 0.8x slower than Gogo ABR, I am wondering if I should test Gogo...

MP3 Listening Test at 128 kbps

Reply #106 –
Last night I saw AlexB's sample thread in the uploads section.

Is this the place for sample proposals, or is it just to provide unknown new samples that can be tested?
lame3995o -Q1.7 --lowpass 17

MP3 Listening Test at 128 kbps

Reply #107 –
Well, that is a rather old thread I started last year, but you can submit samples you think would be valuable for this test. Samples discussion will follow once we have the encoder and the settings set.

MP3 Listening Test at 128 kbps

Reply #108 –
Quote from: Sebastian Mares on
Thanks a lot for your test Alex. I am wondering if the highest iTunes setting (VBR quality 7) would increase bitrate too much. Have to do some testing myself.

I don't have any numbers in the quality options in the English version. Perhaps you have the German version and it is different. Mine has: Lowest, Low, Medium Low, Medium, Medium High, High and Highest.

Quote
By the way, seeing that FhG 4.1 VBR is only 0.8x slower than Gogo ABR, I am wondering if I should test Gogo...

Has anyone verified that FhG 4.1 @ -m 4 & -q 0 would be better than GoGo? (Obviously -m 3 produces too high bitrates for this test.)


Since this encoder discussion seems to not advance I would like to suggest that we discuss about each encoder separately. Perhaps then we could make final decisions. Since you mentioned the iTunes' quality settings we could start with iTunes. I tested some additional iTunes options:



Surprisingly the quality options have almost no effect to the encoding speeds. The very small encoding time differences may be caused by other things as well (I did only one pass). I wonder if these settings do anything, even though there was a very small bitrate increase.

The Smart Encoding Adjustments and Filter Frequencies Below 10 Hz options seem to slightly affect the speed so perhaps they actually do something.

In general I can't see any advantage of using iTunes instead of LAME. In my experience iTunes produces inferior quality and it seems now that it is not significantly faster. Also, it doesn't write a LAME info tag for gapless decoding so even if the quality would be comparable I would prefer LAME. Unless Apple has miraculously increased the quality of this encoder I wouldn't mind if iTunes was dropped from this test. If other forum members insist to have it tested I would suggest using the settings that probably produce the best quality. I think the setting names Apple has chosen suggest that the best quality would be achieved @ 128 kbps VBR, Quality: Highest, Stereo Mode: Joint Stereo, Smart Encoding Adjustments & Filter Frequencies Below 10 Hz: enabled.

Edit: I made a small mistake in the iTunes table. I uploaded a fixed file. Refresh your browser to see it.

MP3 Listening Test at 128 kbps

Reply #109 –
Yes, I only have numbers, 3 of them also having a German description:



I think iTunes is an encoder that should be featured in this test and I would drop Gogo if an encoder really needs to be excluded.

Quote from: Alex B on
Has anyone verified that FhG 4.1 @ -m 4 & -q 0 would be better than GoGo? (Obviously -m 3 produces too high bitrates for this test.)


Shouldn't we test with -q 1?

MP3 Listening Test at 128 kbps

Reply #110 –
Quote from: Sebastian Mares on
I think iTunes is an encoder that should be featured in this test

I've explained why I don't need its test results. Why would you like to see it tested? Just for completeness' sake? In any case this is not a biggie for me, but I think each choice should be supported by proper argumentation. *)

I really would like to see all iTunes's MP3 encoder fanboys (and girls) to show up with their ABX logs and settings suggestions...   

Quote
I would drop Gogo if an encoder really needs to be excluded

So you don't want to discuss about each encoder separately? We have at least three other debates to solve: Helix vs. Real, FhG versions & settings and GoGo vs. the other fastest encoders (which may be an empty list if Real and FhG in the slow mode is chosen). If we could concentrate on one thing at a time perhaps we could make progress.


EDIT

*) Previously I thought that iTunes was faster. Now I have no personal reason for testing it.

MP3 Listening Test at 128 kbps

Reply #111 –
Quote from: Sebastian Mares on
By the way, seeing that FhG 4.1 VBR is only 0.8x slower than Gogo ABR, I am wondering if I should test Gogo...

Quote
Shouldn't we test with -q 1?

These two comments are contradictory. -q 1 is a lot slower than GoGo. We really would need to try a few samples and compare the FhG 4.1 -q 1, -q 0 and perhaps FhG 3.4 quality differences. But as I said I would like to discuss about FhG separately after we are through iTunes.

MP3 Listening Test at 128 kbps

Reply #112 –
OK, you are right about the contradiction - I missed the fact that 71x was with -q 0, sorry.

So here's an important question: do we need to drop any of the 5 contenders LAME, FhG, iTunes, Gogo and Helix? If we add a low anchor to the configuration, there should be 6 encoders.

So here are my reasons for the contenders:

LAME: Recommended encoder here on HA and so far it is said to offer the best quality.

iTunes: Encoder is available for both Windows and OS X and seems to be popular outside HA. People with iPods who prefer to use MP3 over AAC usually choose iTunes to encode. Moreover, some artists who are offering songs in MP3 format seem to use iTunes as encoder (like Bruce Springsteen http://www.radionowheredownload.com/).

FhG: New encoder version which is worth testing because it's also pretty fast.

Helix: Fast encoder.

Gogo: Fast encoder, but on the other hand, quite old and I am wondering if it offers any big advantage over Helix or FhG which at least are actively developed (or is Helix not developed any more?). FhG with -q 1 is much slower that Gogo, though.

MP3 Listening Test at 128 kbps

Reply #113 –
Sounds like the speed thematics is still there as a major one - at least to some of us.

Anyway, I second taking each encoder separately one in a row and call for opinions whether it should be included and with which settings.

And as things began to start already to some extent with iTunes mp3:
I personally like to see it tested, with the settings: q highest, js, smart&filter enabled.
In case average bitrate thus turns out ot be significantly higher than that of the competitors, q medium should be chosen IMO.
lame3995o -Q1.7 --lowpass 17

MP3 Listening Test at 128 kbps

Reply #114 –
Quote from: Sebastian Mares on
iTunes: Encoder is available for both Windows and OS X and seems to be popular outside HA. People with iPods who prefer to use MP3 over AAC usually choose iTunes to encode. Moreover, some artists who are offering songs in MP3 format seem to use iTunes as encoder (like Bruce Springsteen http://www.radionowheredownload.com/).

These are good points. Possibly the test can prove that it is worth of using LAME on a Mac too. It would also be good to be able to show proof to audio techinicians. If iTunes turns out to be better than I expect I would be happy to be proven wrong...

[OT] I have been listening to Bruce's Live 1975-1985 album (3CD Box) for an hour now...  [/OT]

MP3 Listening Test at 128 kbps

Reply #115 –
Before discussing about each encoder separately we should clarify the exact purpose of this test:
1/ evaluating several MP3 implementations at their best settings
2/ evaluating several MP3 implementations at their fastest settings
3/ evaluating several MP3 implementations set with different principles
4/ ???

Once the goal of the upcoming test totally clarified we should debate about the pertinence of each possible competitor.


I would go for the 1st one. The second one is in my opinion anachronic ; the third one would be biased (as it supposes that we can arbitrary decide that some implementations are only worth for their speed and don't have to be evaluated at their full quality potential).
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz
 

MP3 Listening Test at 128 kbps

Reply #116 –
I asked the same question a few posts ago and it seems that people tend to want 1. However, if the number of contenders is a problem, we need something in order to choose whether or not encoder A should be preferred over B. In such a situation, I think speed and popularity could be good criterions.

MP3 Listening Test at 128 kbps

Reply #117 –
I go with 1) as well.

I'm also interested in the behavior of different settings, but as most people do I see that this is not easy to do and we better don't.

Quote from: Sebastian Mares on
... if the number of contenders is a problem ...

5 contenders and 1 low anchor - is that too much?
lame3995o -Q1.7 --lowpass 17

MP3 Listening Test at 128 kbps

Reply #118 –
Quote from: Sebastian Mares on
I asked the same question a few posts ago and it seems that people tend to want 1. However, if the number of contenders is a problem, we need something in order to choose whether or not encoder A should be preferred over B. In such a situation, I think speed and popularity could be good criterions.

I agree: popularity seems a good criterion (though it's sometimes hard to evaluate).

I'm also interested in the behavior of different settings, but as most people do I see that this is not easy to do and we better don't.
Do you mean testing different settings of the same encoder? It's interesting indeed, but this task belong to a dedicated listening test.
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

MP3 Listening Test at 128 kbps

Reply #119 –
I thought it was obvious why I and some others are interested about the faster options. We already have found out that LAME -V5 produces reasonably fine quality in the multiformat test. For many of us the LAME Info tag is very important and we would like to see its support extented to all possible HW & SW players. IMO, the HA members would need very strong reasons to use something else than LAME.

Personally, I have my ripped CDs archived in a lossless format. I have one copy of the complete archive in a high bitrate lossy format for home hifi usage. For portables I would like to encode smaller files on the need basis.

Normally I can do that with LAME, but there are times when I am for example about to start a holiday trip in about two hours and I would like to encode 30 hours of new music to take with me. (I know, I should have done that earlier, but sometimes things just don't go as planned.)

If the decoding speed from the lossless source is 50x and LAME encodes at 15x speed it would take about 2 hours 40 minutes. If a faster encoder encodes at 75x speed the process would take one hour.

The question is, if the quality of the used fast encoder is acceptable.

Edit: typos...

MP3 Listening Test at 128 kbps

Reply #120 –
Alex B> I'm sometimes in the same situation (the need to encode quickly some lossless files). I also noticed that the effective speed of fast encoders may be drastically ruined by fragmentation and disk access (amplified by multi-core processors).

I'm also quite sure that the whole process lossless library -> MP3 isn't the most popular way to get MP3 files (most people are probably ripping CD to MP3 and in this case the encoding speed is usually limited by the extraction time). And as you said, speed is only needed in some specific situation (before holidays, etc... otherwise overnight encoding with best/slow encoders is fine). So is this marginal (I suppose) behaviour enough to justify a public listening test? Or shouldn't we use Sebastian's opportunity to evaluate for once the real potential of most popular/attractive LAME competitors (which seems to be all considerably faster than LAME)?
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

MP3 Listening Test at 128 kbps

Reply #121 –
Perhaps the encoder/settings selection doesn't have to be based on the speed or on the quality. Let me explain.

In my speed test I included LAME -V5 -g5. It wasn't significantly faster. In my experience LAME -q5 at medium bitrates is the lowest usable q setting. So for LAME I think it is better to use the standard -V5 setting.

As we have seen iTunes doesn't have any fast VBR settings, so it is best to select the highest quality setting.

The choice between Helix and RealPlayer may not be difficult after all. We have no knowledge about the internal options used in RealPlayer and it doesn't seem to have any speed advantage. The RealPlayer UI is awkward and the converter feature is not available in the free version. A new RealPlayer version is on the beta stage, but I have no knowledge about it (other than it looks different). I would select the more versatile Helix CL encoder and use a suitable -V setting without any additional untested quality switches. However, it would be good to check if Helix has any newer source code available. Do we have any voluntary developers who could create a new compile if needed?

IMHO, the FhG settings should be quickly pre-tested. The difference between -q 0 and -q 1 can be anything. We don't even know if -q1 is better at all or for example if -q 0 is below all usability -- and we will never know that if don't try it before choosing the setting. I can contribute.

That leaves us GoGo. I could include it in my pretest. I hope some others would be able to do the same.

I would like to suggest the following: If GoGo is clearly worse than FhG -q 0 and -q 1 then we could drop it from the test and include both FhG settings (in case both settings seem to be promising). If GoGo seems to be able to provide useful or even better quality than FhG we could continue this discussion.

Also, I think 5+1 would be fine.

MP3 Listening Test at 128 kbps

Reply #122 –
Very reasonable and practicable procedure IMO.
lame3995o -Q1.7 --lowpass 17

MP3 Listening Test at 128 kbps

Reply #124 –
Might I ask as to why you seem to have stowed away your idea of last spring to organize a (yet unexplored) mid-bitrate multi-codec test next, after my inquiry about a 96k test:
http://www.hydrogenaudio.org/forums/index....53134&st=78

I can't help but wondering if it isn't a bit of a waste of time and resources to explore 128k MP3 (once again), probably the most extensively tested/known/used codec setting today, while the sub-100k range almost remains terra incognita in the field of public listening tests.

Edit: not to mention, as was coined in this thread already, that the prospect of getting all-tied 4.5ish results, seems hardly challenging or even interesting to me.