HydrogenAudio

Lossy Audio Compression => MP3 => MP3 - Tech => Topic started by: halb27 on 2012-12-18 00:11:07

Title: lame3100h, a functional extension
Post by: halb27 on 2012-12-18 00:11:07
You can download it from here (http://dl.getdropbox.com/u/2681777/Lame3100h/lame3100h.zip).

What’s the functional extension?

It offers additional VBR quality settings -V5+ to -V0+ which cover the average bitrate range from ~144 to ~300 kbps (for a variety of pop music):

-V5+     144 kbps
-V4.5+  160 kbps
-V4+     176 kbps
-V3.5+  192 kbps
-V3+     208 kbps
-V2+     224 kbps
-V1.5+  240 kbps
-V1+     256 kbps
-V0.5+  278 kbps
-V0+     300 kbps

An alternative way to use the functional extension is --brV+ x, where x is the average bitrate (for a variety of pop music) you want to use. You can use for instance --brV+ 224 instead of -V2+.

What is it good for?

Lame’s moderate VBR quality settings like -V5 or -V4 usually yield a very good quality. That’s why many users are happy with these settings. Sometimes however tracks contain spots which are not encoded well. Many users want a better quality also for these rather rare events. From current experience Lame3.100 alpha2 seems to scale well quality of tonal problems with -Vn level, but temporal resolution can still be an issue.

-Vn+ uses -Vn as the encoding basis, but adds a certain amount of brute-force safety by forcing audio data bitrate to a target bitrate which depends on -Vn+ level. Moreover care is taken to always provide maximum possible audio data space for the encoding of short blocks which are used when the encoder thinks it is appropriate for a good temporal resolution. Also Lame's default lowpass is lowered a bit in order to make best use of the encoded bits (use --lowpass x if you don't like this).

Emphasis is on issues with temporal resolution, but tonal problems are tackled as well.

In a sense -Vn+ combines the quality advantages of both VBR and CBR.

Recommendations

Users who care much about filesize and are content with the functional extension improving short block (pre-echo) behavior, can use -V5+ to -V4+.

Users who don’t like rather obvious issues in their music even when they’re rare but who also care about filesize are best to choose from -V3.5+ to -V1.5+ according to their needs.
For a significant potential for improving tonal issues -V3+ or better is recommended.

Users who don’t care much about filesize but much more about universal top quality are best served by using -V1+  or V0+, or anything in between.

Installation

lame3100h.exe was compiled with Visual C++ 2010. For this reason it is necessary to install the Microsoft Visual C++ 2010 Redistributable Package vcredist_x86.exe. You can download it from  http://www.microsoft.com/en-us/download/details.aspx?id=8328 (http://www.microsoft.com/en-us/download/details.aspx?id=8328)

lame3100h.exe uses the fast and lossless mp3packer tool internally to squeeze the otherwise unused bits out of the mp3 file. You can download mp3packer from  http://www.hydrogenaudio.org/forums/index....st&p=282289 (http://www.hydrogenaudio.org/forums/index.php?showtopic=32379&view=findpost&p=282289). Put mp3packer.exe into the same folder where lame3100h.exe is located. Many thanks to Omion for this great tool.
In case there is no mp3packer.exe in lame3100h.exe’s folder lame3100h.exe will work, but the mp3 files will be somewhat larger than necessary.
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2012-12-18 04:38:06
Absolutely great, halb27! Thank you very much for your effort!
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2012-12-18 18:53:17
Average bitrate on my set of pops and jazz albums.
Code: [Select]
lame3100h --brV+ %v %i %o
%v   Average bitrate[bps]
0    268195
5    133357
9    72714
10    32362
143    32362
144    143518
160    159543
176    175258
192    191156
208    206293
224    222684
240    239031
256    254897
272    270994
288    286859
300    297909
301    32362
320    32362
329    32362

Title: lame3100h, a functional extension
Post by: BFG on 2012-12-18 22:05:20
Average bitrate on my set of pops and jazz albums.

Looks like it's working as advertised!  If it's possible to do so, I'd recommend extending the existing algorithm so that -brV+ 32 through -brV+ 320 are also recognized.  I know that -V+ 32-143 isn't particularly useful, but I could see uses for 301-320.
Title: lame3100h, a functional extension
Post by: Dynamic on 2012-12-19 02:05:32
Average bitrate on my set of pops and jazz albums.
%v  Average bitrate[bps]
0   268195
5   133357
9   72714
10   32362
143   32362

144   143518
160   159543
176   175258
192   191156
208   206293
224   222684
240   239031
256   254897
272   270994
288   286859
300   297909
301   32362
320   32362
329   32362


So those that I've bolded are not close to the bitrate requested. I guess anything below 143 should ideally be mapped to 144 and anything above 300 should be mapped to 300 and possibly throw a warning that the requested bitrate target is out of range and specify the one actually used.
Title: lame3100h, a functional extension
Post by: BFG on 2012-12-19 03:12:55
So those that I've bolded are not close to the bitrate requested. I guess anything below 143 should ideally be mapped to 144 and anything above 300 should be mapped to 300 and possibly throw a warning that the requested bitrate target is out of range and specify the one actually used.

It looks like the new --brV+ function is using the standard -V+ function for values 0-9, and anything outside of the 144-300 range (the range that -V+ is defined for) errors out.  Either 32-143 should be mapped to 144, and 301-320 to 300 like Dynamic suggests, or the --brV+ function should be extended to the full range of MPEG-1 bitrates (32-320).
Title: lame3100h, a functional extension
Post by: halb27 on 2012-12-19 10:19:23
This is very strange.
I reject --brV+ values lower than 144 and higher than 300 kbps and stop with an error message.
I've downloaded the published version again, and it behaves as it should here.

Has somebody else encountered the behavior Kamedo2 ran upon?
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2012-12-19 15:27:32
Sorry, I forgot to delete past mp3s and the results were a combination of
lame3100h -brV%v %i %o
lame3100h --brV+ %v %i %o
The only 144-300 section is the result of --brV+ %v.
Code: [Select]
lame3100h --brV+ %v %i %o
144 143518
145 150712
146 151222
147 151723
148 152186
149 152730
150 153276
151 153857
152 154381
153 154908
154 155439
155 156028
156 156618
157 157186
158 157809
159 158478
160 159543
161 160236
162 160974
163 161754
164 162568
165 163390
166 164271
167 165197
168 166145
169 167159
170 168224
171 169293
172 170447
173 171554
174 172759
175 173987
176 175258
177 176207
178 177174
179 178172
180 179124
181 180137
182 181121
183 182142
184 183136
185 184107
186 185079
187 186076
188 187113
189 188099
190 189082
191 190092
192 191156
193 192105
194 193085
195 194026
196 194903
197 195916
198 196857
199 197721
200 198653
201 199644
202 200595
203 201577
204 202464
205 203397
206 204396
207 205374
208 206293
209 208505
210 209517
211 210492
212 211328
213 212258
214 213229
215 214046
216 215058
217 216060
218 216894
219 217896
220 218912
221 219829
222 220857
223 221780
224 222684
225 223693
226 224617
227 225739
228 226710
229 227728
230 228821
231 229792
232 230682
233 231673
234 232697
235 233665
236 234751
237 235687
238 236712
239 237782
240 239031
241 240096
242 241108
243 242169
244 243188
245 244241
246 245264
247 246268
248 246957
249 247980
250 249062
251 250053
252 251081
253 252074
254 253103
255 254140
256 254897
257 255971
By the way, I prefer the --brV+ 320 to be redirected to --brV+ 300, meaning -V0+, because the 320 kBit/s bitrate selection box on EAC V1.0b3 can be used without errors.
Title: lame3100h, a functional extension
Post by: BFG on 2012-12-19 19:56:16
By the way, I prefer the --brV+ 320 to be redirected to --brV+ 300, meaning -V0+, because the 320 kBit/s bitrate selection box on EAC V1.0b3 can be used without errors.

Personally, I would prefer that --brV+ 320 actually aim for ~320kbps, or result in an error code, rather than be redirected.  But this is a simple preference.
Title: lame3100h, a functional extension
Post by: halb27 on 2012-12-19 20:23:03
I've changed --brV+ x behavior for x < 144 and x > 300.
No more error message and stop, instead: --brV+ 144 is used for x < 144, --brV+ 300 is used for x > 300.
You'll get this behavior when downloading now.

The --brV+ x -> -V y+ mapping isn't very good when x is a little bit higher than 144.
The background is that -V5 yields an average bitrate of 130 kbps, wheras -V4.99 takes an average bitrate of 140 kbps. This bitrate jump is at least softened when going -Vx+.

Title: lame3100h, a functional extension
Post by: Kamedo2 on 2012-12-20 14:59:47
I've changed --brV+ x behavior for x < 144 and x > 300.
It worked for me. Thank you!
Code: [Select]
lame3100h --brV+ %v %i %o
%v  average bitrate of pops and jazz albums[bps]
0 143518
4 143518
8 143518
12 143518
16 143518
20 143518
24 143518
28 143518
32 143518
36 143518
40 143518
44 143518
48 143518
52 143518
56 143518
60 143518
64 143518
68 143518
72 143518
76 143518
80 143518
84 143518
88 143518
92 143518
96 143518
100 143518
104 143518
108 143518
112 143518
116 143518
120 143518
124 143518
128 143518
132 143518
136 143518
140 143518
144 143518
148 152186
152 154381
156 156618
160 159543
164 162568
168 166145
172 170447
176 175258
180 179124
184 183136
188 187113
192 191156
196 194903
200 198653
204 202464
208 206293
212 211328
216 215058
220 218912
224 222684
228 226710
232 230682
236 234751
240 239031
244 243188
248 246957
252 251081
256 254897
260 259052
264 263170
268 266899
272 270994
276 275071
280 278791
284 282878
288 286859
292 290465
296 294294
300 297909
304 297909
308 297909
312 297909
316 297909
320 297909
324 297909
Title: lame3100h, a functional extension
Post by: MrJAck1983 on 2012-12-21 14:08:29
When do you guys think you will release lame 3100 officially ?
Title: lame3100h, a functional extension
Post by: BFG on 2012-12-21 14:26:18
When do you guys think you will release lame 3100 officially ?

Jack,
Halb27 isn't an official author of LAME - he just has a popular extension, based on the main version of LAME, that he releases to HydrogenAudio enthusiasts regularly.
Title: lame3100h, a functional extension
Post by: MrJAck1983 on 2012-12-21 16:57:28
When do you guys think you will release lame 3100 officially ?

Jack,
Halb27 isn't an official author of LAME - he just has a popular extension, based on the main version of LAME, that he releases to HydrogenAudio enthusiasts regularly.

I understand now.. thank you for the clarification!
Title: lame3100h, a functional extension
Post by: lunasnow on 2012-12-27 14:14:43
if I use Lame

add -q0 or -h parameter can provide better quality ?

Should I use -q0 or -h setting?
Title: lame3100h, a functional extension
Post by: [JAZ] on 2012-12-27 16:55:23
Replied at the other thread (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=98512&view=findpost&p=818538)
Title: lame3100h, a functional extension
Post by: o-l-a-v on 2013-01-02 11:02:54
With command
Code: [Select]
-S --noreplaygain -V1+ -%d

The progressbar does not move, estimated time just get higher and higher. Whats wrong here?
I use foobar. I got mp3packer.exe in the same folder, and haven't renamed the lame3100h.exe.
Also got c++ 2010 installed
Title: lame3100h, a functional extension
Post by: lvqcl on 2013-01-02 11:05:06
there should be a space between - and %d
Title: lame3100h, a functional extension
Post by: o-l-a-v on 2013-01-02 11:27:55
there should be a space between - and %d


Thanks, that was it..
Title: lame3100h, a functional extension
Post by: goa pride on 2013-01-03 11:33:48
what's the benefit using this lame3100h 32 bit only instead of lame 3.100 alpha2 64 bit preset V0 ?
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2013-01-07 13:43:26
I'm almost ready for the MP3 224kbps ABC/HR listening tests.
Encoders
LAME3100h(this encoder) V2+
http://dl.getdropbox.com/u/2681777/Lame3100h/lame3100h.zip (http://dl.getdropbox.com/u/2681777/Lame3100h/lame3100h.zip)
LAME 3.99.5 V1
http://www.rarewares.org/files/mp3/lame3.99.5.zip (http://www.rarewares.org/files/mp3/lame3.99.5.zip)
LAME 3.98.4 CBR 224kbps -q 0

Helix MP3 Encoder(2005) -X2 -U2 -V146
http://www.rarewares.org/files/mp3/helix_mp3enc_CVS.zip (http://www.rarewares.org/files/mp3/helix_mp3enc_CVS.zip)
BladeEnc 0.94.2(low anchor)
http://www.softpedia.com/get/Multimedia/Au.../BladeEnc.shtml (http://www.softpedia.com/get/Multimedia/Audio/Audio-CD-Rippers-Encoders/BladeEnc.shtml)

Bitrates
encoders 25-samples albums
3100h  234.6  222.7
3.995  227.5  224.6
3.984  224.6  224.1
Helix  221.3  225.3
Blade  224.1  224.0

Decoder: Madplay(32bit decoding)

Should I wait for something newer or fix anything?
Title: lame3100h, a functional extension
Post by: Aleron Ives on 2013-01-07 20:10:16
Why are you comparing VBR with 3.99.5 and 3.100h against CBR with 3.98.4? If you want the target bitrates to be close to 224 kbps for comparison purposes for each version, it seems to me that you ought to use V 1.5 or some other non-standard value in 3.98.4 so you're comparing VBR against VBR in all cases, rather than quality-based encoding schemes against a filesize-based one. Perhaps I have misunderstood the purpose of your test.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-01-07 20:40:56
what's the benefit using this lame3100h 32 bit only instead of lame 3.100 alpha2 64 bit preset V0 ?

You can expect quality improvements of 3100h -V0+ over 3.100a2 -V0 on rare occasion. It's up to you whether you think it's worth it.
I will stay with only the 32 bit version for reasons of simplicity.

Sorry for the late reply.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-01-07 20:52:57
Should I wait for something newer or fix anything?

As far as lame3100h is concerned: I have no plans for a newer version.
But I see you don't include Lame 3.100 alpha2 in the test. This brings the problem that in case lame3100h would show up quality advantages in your test we don't know whether these are due to the improvements of 3.100 or whether they are due to the functional extension.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-01-07 21:02:14
Why are you comparing VBR with 3.99.5 and 3.100h against CBR with 3.98.4? ...

I find it very interesting to have a CBR 224 version for comparison. Quality behavior is different, and it would be great to learn about these differences in a new listening test IMO.
Title: lame3100h, a functional extension
Post by: SubV on 2013-01-14 19:01:00
I've compiled the LAME3100h with SSE2 support, which results in much faster encoding.

Also included the VC++ 2010 dll's and the latest version of mp3packer.

You can download it here (https://www.box.com/s/978zaw2x4p9omz7wojcy).
Title: lame3100h, a functional extension
Post by: FreaqyFrequency on 2013-01-14 21:13:04
Awesome!  Spasibo Sub! 
Title: lame3100h, a functional extension
Post by: lock67ca on 2013-01-15 02:34:11
So there's such an improvement over 3.99.5 that you're really comfortable using an alpha version for archiving? That does sound promising but I'm not sure I want to make the jump until there's an official release.
Title: lame3100h, a functional extension
Post by: BFG on 2013-01-15 05:00:12
So there's such an improvement over 3.99.5 that you're really comfortable using an alpha version for archiving? That does sound promising but I'm not sure I want to make the jump until there's an official release.

Perhaps I'll regret it later, but I've already ripped 80 CDs using 3.100h, and have no regrets so far.  It's been equal or superior to 3.99.5 in every listening test I've done so far.  And it does notably reduce preecho.
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2013-01-15 14:14:54
I used both halb27's original binary and SubV's sse2-enabled compile, and I've got slightly different results, both encoded mp3 and decoded wav.
-V2+, 5 min snippet of pops and jazz musics.
halb27    SubV
8350652 8349508 filesize[Byte]
.049972 .049981 similarity

halb27's binary is producing 0.014% larger mp3 with slightly better accuracy, at least on the waveform.
I don't care about these subtle difference, but I report it anyway.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-01-15 16:35:00
From the Wikipedia on SSE2 I see differences are to be expected. Precision of SSE2 calculations can be reduced. Whether this has an audible impact is another question.

I personally want to provide only exes which hopefully can be used on any Windows system, especially as to me it's fast enough using foobar and a multi-core system (on my 6-core HT system 12 tracks are encoded in parallel). But I welcome any variant like this one for those who want  to use it.
Title: lame3100h, a functional extension
Post by: SubV on 2013-01-21 12:09:17
I used both halb27's original binary and SubV's sse2-enabled compile, and I've got slightly different results, both encoded mp3 and decoded wav.
-V2+, 5 min snippet of pops and jazz musics.
halb27    SubV
8350652 8349508 filesize[Byte]
.049972 .049981 similarity

halb27's binary is producing 0.014% larger mp3 with slightly better accuracy, at least on the waveform.
I don't care about these subtle difference, but I report it anyway.

http://www.hydrogenaudio.org/forums/index....st&p=819634 (http://www.hydrogenaudio.org/forums/index.php?showtopic=98689&view=findpost&p=819634)

Quote from: db1989 link=msg=0 date=
Binaries compiled with different settings/optimisations/etc. may produce insignificantly differing bitstreams. This is a non-issue in reality.

Title: lame3100h, a functional extension
Post by: BFG on 2013-01-21 16:16:23
Would it be possible to build a single 3.100h binary which is able to detect if SSE2 is available on the user's machine, and if so, enables it by default?  (You'd want to add a flag that can force-disable or force-enable, most likely.)  This would be better than having two different binaries.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-01-21 16:47:38
As far as I can see in the VC++ environment you just have the option to use configuraton 'release' or 'releaseSSE2'. So this is a static decision while developing the software.
Title: lame3100h, a functional extension
Post by: [JAZ] on 2013-01-21 18:49:38
halb27 is correct (although the naming doesn't matter).

Most compilers only compile in one setting. (i.e. targeting a supported instruction set, like SSE, or not).

Back in the days, there were software that had different paths, because of hand-written code (i.e. writing assembly code by hand).  LAME used to have an assembly path, but with 64bits, "old" assembly doesn't even compile, so developers have to either use intrinsics, or maintain (again) separated branches.

A semi-automatic way could be done by the use of dlls, (each one built with one support, and loading the one needed), but again, that adds work to the developer.

And also, there's the "bundle" solution.  Like the "universal binary" in Apple's OS or the binary of "process explorer" (From Microsoft), which are actually two complete binaries built in each setting, wrapped inside a binary that knows when to use one or the other.

Title: lame3100h, a functional extension
Post by: halb27 on 2013-01-23 14:15:35
It's not a long time ago that I wrote 'I have no plans for a new version'. Well, things can change fast. A few days ago I listened carefully to Camille's 'Là où je suis née' which includes a spot with a tiny issue (see this thread from 3.98beta days (http://www.hydrogenaudio.org/forums/index.php?showtopic=60114&view=findpost&p=538321)). Quality has improved IMO with the newer Lame versions, but I was surprised that it's still a subtle issue. To be quite clear: I am not able to ABX it on a regular basis, but I can when I am in a good condition. I guess it wouldn't worry me if it was a spot in a say hard harpsichord sample, but this is 'just' female voice. And worse for me: my impression is that lame3100h -V0+ is inferior to plain lame3.100.a2 -V0. Sorry for being vague; the issue is really subtle to me, and maybe I'm just too nitpicking.

Nitpicking has its merits though, and looking up the problem I see that this is one of those cases where we're running out of channel and granule space. This was a reason for thinking things over, and I found that my target bitrate control can be improved. Target bitrate control is with respect to frames at the moment, but I can do it also on a granule or even channel basis. This may improve Camille's song or not, but I feel I should do it anyway. It is the more adequate way of doing it for frames of mixed block type (start/short, short/stop), as at the moment the granule of start resp. stop type gets too many bits in such a mixed block situation. Improving this means improving short block behavior a bit. So I will do it.

Sorry to everybody who has started testing, especially Kamedo2. I suggest to just continue testing with lame3100h.
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2013-01-24 17:15:42
halb27, Is the new version available? I'm considering discarding current results(only 2) and testing the new one. More people will be interested if it's the latest encoder.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-01-24 19:30:39
I'm working on the new version. I guess it will take me a week or so for developing, and another week for testing.
Testing will be a bit of a problem because the changes of the new version are pretty substantial. After first tests of my own I plan to publish a prerelease, so hopefully some members will contribute to testing.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-05 16:23:08
Here (http://dl.getdropbox.com/u/2681777/Lame3100i/lame3100i.zip) comes version 3.100i for testing.
I've also included a SSE2 version, just for experience, I am not happy with it: eig_essence is clearly worse to me than when encoded with the non-SSE2 version. So I probably will not include it in the completely tested version I'll publish in a separate thread.

Samples for testing can be downloaded from here (http://dl.getdropbox.com/u/2681777/Lame3100i/samples.zip). Of course you're welcome to do any tests you like.

Controlling quality of the granules (halfs of the frames) instead of frames as I do with this version for the very first time has proven to be essential. The major reason why there is an audible issue at ~sec. 3.0 of eig with original Lame is frame #109 which contains granules of type 'short' and 'stop'. The 'stop' granule needs a lot of bits. With 3100h it gets a lot of bits, but only because the frame has a short granule, and target bitrate control is on a frame basis. I added a bit of silence to the beginning of the track so that this stop granule goes into frame #110 (now a stop/start frame). Now 3.100h doesn't help much, and at least with the lower quality -Vn+ levels the issue is very audible. This modified version of eig_essence is included in the samples zip file described above.

As I wrote earlier 3.100i means a lot of changes compared to 3.100h. I'd be very happy if some of you could contribute with testing.
Title: lame3100h, a functional extension
Post by: Jeff Flowerday on 2013-02-05 19:59:50
As I wrote earlier 3.100i means a lot of changes compared to 3.100h. I'd be very happy if some of you could contribute with testing.


Using dbpoweramp batch converter, I'm recreating my 120,000 track mp3 library from my source flacs using -V0+.  It will take a few days.  I guess this is a test in it's own.


Sure, I'll probably have to repeat the process in a few weeks.  I wasn't completely happy with my 3.99.5 -V0 files so I don't mind allocating the processing power to the task even if it's temporary.
Title: lame3100h, a functional extension
Post by: Jeff Flowerday on 2013-02-05 20:59:50
It is to bad the SSE compiled version doesn't seem to create the same results.  The non SSE version is slow as molasses in comparison.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-05 21:25:03
Thanks for testing. But wouldn't it be a good idea to first test those tracks you were not totally happy with when using 3.99.5 -V0 before going to encode a huge collection with this prerelease version?

The SSE2 result must be necessarily different from the non-SSE2 one due to the different arithmetics. Not bad per se, but as I wrote my experience isn't a good one.
At least on my system the non-SSE2 version is not very much slower, something like ~145x (non-SSE2) against ~190x (SSE2) according to foobar.
Title: lame3100h, a functional extension
Post by: Jeff Flowerday on 2013-02-05 22:01:48
Thanks for testing. But wouldn't it be a good idea to first test those tracks you were not totally happy with when using 3.99.5 -V0 before going to encode a huge collection with this prerelease version?

The SSE2 result must be necessarily different from the non-SSE2 one due to the different arithmetics. Not bad per se, but as I wrote my experience isn't a good one.
At least on my system the non-SSE2 version is not very much slower, something like ~145x (non-SSE2) against ~190x (SSE2) according to foobar.


I did do some comparisons with tracks I'm vary familiar with.  Not scientific by any means but I liked what I heard or maybe thought I heard. My general observations don't really fit into the scientific nature of this forum.


Your right the difference isn't that large between your i versions.  I don't know what version I was comparing it to, I've been tossing in different lame.exe files left and right this AM.  That said a 33% increase in performance does add up when dealing with my volume of tracks.
Title: lame3100h, a functional extension
Post by: IgorC on 2013-02-06 22:10:18
Have tried SSE2 and non SSE2 version at V3+ and couldn't hear any difference between them on eig_essence sample.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-06 22:18:17
I heard the difference at second ~3.0, and I used -V0+ or -V0.7+ (don't remember exactly and can't redo the test at the moment because I would disturb my wife).
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-07 09:49:54
I just retried using my favorite setting -V0.7+, and around sec. 3.0 the difference is very audible to me.
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2013-02-07 16:40:44
I encoded all of my 25 samples and test albums using both sse2_enabled and non-sse 3100i encoder.
Non-sse version had 0.017% more accurate equalized waveform, using 0.0005% more bitrates.
The sample that yielded the most accuracy difference was 15. davinci(speech), with non-sse was 0.25% more accurate.
The sample that yielded the most bitrate difference was FloorEssence(techno), with non-sse was 0.03% bigger.

I used V2+. This is a very superficial test, which counts only overall fidelity to the original and filesize.
So even if the overall difference is only 0.25%, it can have many audible effects.

Title: lame3100h, a functional extension
Post by: Kamedo2 on 2013-02-08 21:42:15
halb27, would you please get rid of the audible difference between the standard and sse2-enabled encoder?
It should be nice if the two outputs almost the same results, because no two separate listening tests are needed to assess its quality.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-08 22:44:57
Sorry, I have no idea how to do that. I rather consider the SSE2 version having been worth a try, but IMO being not worth to be used.
Title: lame3100h, a functional extension
Post by: IgorC on 2013-02-09 02:47:36
You're right, something is wrong with SSE2 version.
Some strong pre- or post-echo artifact at approx. 2.8-3.0 second.


Code: [Select]
ABC/HR Version 1.1 beta 2, 18 June 2004
Testname:

1R = D:\Audio\3100i\eis essense\eig_essence LAME V0.7+.wav
2L = D:\Audio\3100i\eis essense\eig_essence LAME V0.7+ SSE2.wav

---------------------------------------
General Comments:

---------------------------------------
1R File: D:\Audio\3100i\eis essense\eig_essence LAME V0.7+.wav
1R Rating: 4.0
1R Comment:
---------------------------------------
2L File: D:\Audio\3100i\eis essense\eig_essence LAME V0.7+ SSE2.wav
2L Rating: 3.0
2L Comment:
---------------------------------------
ABX Results:
D:\Audio\3100i\eis essense\eig_essence LAME V0.7+.wav vs D:\Audio\3100i\eis essense\eig_essence LAME V0.7+ SSE2.wav
    5 out of 5, pval = 0.031

In my opinion, people who use your encoder  first of all care about quality and then speed. So if SSE2 causes some
precision loss then it might be worth to drop this optimization. Though it's quite strange that it causes such distortion  because untill now there is a common belief that different compilers shouldn't produce noticeable audible differences.

I get a miserable 6x speed (non-SSE2 3.100i) on my Atom based netbook with all 4 threads fully loaded  , but it's not an issue at all.
Title: lame3100h, a functional extension
Post by: SubV on 2013-02-10 13:05:23
In my opinion, people who use your encoder  first of all care about quality and then speed. So if SSE2 causes some
precision loss then it might be worth to drop this optimization. Though it's quite strange that it causes such distortion  because untill now there is a common belief that different compilers shouldn't produce noticeable audible differences.

I get a miserable 6x speed (non-SSE2 3.100i) on my Atom based netbook with all 4 threads fully loaded  , but it's not an issue at all.

AFAIK, most LAME builds from Rarewares are SSE2-optimized, too (and because of that, these builds are very fast). The precision loss appears (most likely) because LAME 3.100 is in early alpha stage now. In my opinion, we'll need to wait for a stable LAME release.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-10 20:44:26
Before publishing 3100i I will remove the SSE2 version because I am not able to do anything about it no matter what's the reason for the problem.

As for the non-SSE2 version which is about to go final:
My sensitivity for pre-echo stuff has improved a little bit, and based on eig I have tried to optimize the precision of the impulses so that I personally am rather happy with 3100i's behavior. I'm well aware though that this doesn't tell very much about audible quality for pre-echo sensitive people or maybe average Joe.
I'd welcome very much if somebody please could check if there can be improvement for critical stuff by using other values for --adbr_short (default: 510 kbps for -V1+ or better) and/or --adbr_start (default: 260 kbps for -V1+ or better).

Added: I forgot that maybe the --adbr-stop defaults (>= 380 kbps for -V1+ or better) might be a bit high for the case that a short/short frame is followed by a short/stop frame, because in this case the stop granule may eat up too much of the available data space so that the short granule before it does not get enough bits.
So trying a somewhat lower --adbr_stop value may be worth while too.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-11 13:39:15
I consider the --adbr-stop behavior I mentioned a bug, because it's my strategy to favor short blocks in a situation of restricted available data space.
I corrected this, and you will get this behavior when downloading (http://dl.getdropbox.com/u/2681777/Lame3100i/lame3100i.zip) again. No need to test other --adbr_stop values (at least not for me) with this version.
I also deleted the troublesome SSE2 version.
Title: lame3100h, a functional extension
Post by: lock67ca on 2013-02-11 23:44:04
I'm still not completely comfortable using an alpha version of Lame for anything beyond testing purposes. I do like what I've heard so far though. Is the last version of the 99.5 extension (I think it was 995f) considered stable enough for use?
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-12 01:40:35
Yes, 3.99.5f was the last 3.99.5 extension version. No stability issues were reported, all the younger versions were because of the arrival and good experience so far with 3.100a2, and due to further ripening concerning the extension.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-13 14:20:14
I've done another small change to 3100i concerning behavior of frames of block type short/short in situations where the encoder is running out of data space. Should be a bit more appropriate now in certain situations. Available when downloading (http://dl.getdropbox.com/u/2681777/Lame3100i/lame3100i.zip) again.
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2013-02-15 00:11:00
I'll retest, because the result of the latest encoder is more interesting. Expect to take 150 days or so.
Encoders
LAME3100i(this encoder) V2+
http://dl.getdropbox.com/u/2681777/Lame3100i/lame3100i.zip (http://dl.getdropbox.com/u/2681777/Lame3100i/lame3100i.zip)
LAME 3.99.5 V1
http://www.rarewares.org/files/mp3/lame3.99.5.zip (http://www.rarewares.org/files/mp3/lame3.99.5.zip)
LAME 3.98.4 CBR 224kbps -q 0

Helix MP3 Encoder(2005) -X2 -U2 -V146
http://www.rarewares.org/files/mp3/helix_mp3enc_CVS.zip (http://www.rarewares.org/files/mp3/helix_mp3enc_CVS.zip)
BladeEnc 0.94.2(low anchor)
http://www.softpedia.com/get/Multimedia/Au.../BladeEnc.shtml (http://www.softpedia.com/get/Multimedia/Audio/Audio-CD-Rippers-Encoders/BladeEnc.shtml)

Bitrates
encoders 25-samples albums
3100i  235.0  222.9
3.995  227.5  224.6
3.984  224.6  224.1
Helix  221.3  225.3
Blade  224.1  224.0

Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-15 08:31:36
OK, so I publish the current 3100i as final in a new thread, as I have not heard of anybody currently doing a listening test and I have finished all my testing.
Title: lame3100h, a functional extension
Post by: shadowking on 2013-02-25 13:49:24
I started to test 3.99.5 V1 V0+, 320k as well as the funtional extension builds (3995f). Quality is pretty good with both but unfortunately some problems remain and some are pretty bad.

Emese: Very bad quality up to V1/ V1+ and still not too good at V0/V0+. No benefit from the + settings. Tested 3100i - improvement. V0+ is still a bit noisy.

Florida Seq : To me its way worse than EIG. At 5.0 secs the smearing is so severe. I abx V0/V0+ 10/10  without listening to the reference. The + settings provide no benefit here.  Edit: I tried again with 3100i: 7/10 then 10/10. I think there is an improvement but still a problem.

Angel falls 1st : Good progress with the + settings. Ringing is much reduced. + Settings have big impact here.

EIG: There is a click maybe around 3 secs easily heard @ V3 and V2. Much reduced at V1 / V0. Otherwise pretty good.

Catanets2_edit: 3.99.5 CBR modes abxable including 320k  10/10.. VBR is much better failed to abx V0
Title: lame3100h, a functional extension
Post by: Kamedo2 on 2013-02-25 17:57:20
I started to test 3.99.5 V1 V0+, 320k as well as the funtional extension builds (3995f). Quality is pretty good with both but unfortunately some problems remain and some are pretty bad.

Did you use an ABC/HR software?
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-25 18:32:46
@shadowking:

Thank you very much for testing. Sounds like 3.100a2 and/or 3.100i are worth the development.
Sure there are samples that need highest quality settings or can't even be brought close to perfection with the highest settings.
I'll have a look at Emese and Florida Seq.
Title: lame3100h, a functional extension
Post by: shadowking on 2013-02-26 02:43:27
I tried again 1st thing this morning with fresh ears 3100i V0+

Florida: I didn't compare to 3.99.5 directly but have a strong feeling that 3100 is better (florida). Still easy to abx

foo_abx 1.3.4 report
foobar2000 v1.0.3
2013/02/26 11:37:50

File A: C:\stuff\music\abx\florida_seq.wav
File B: C:\stuff\music\florida_seq.mp3

11:37:50 : Test started.
11:38:10 : 01/01  50.0%
11:38:15 : 02/02  25.0%
11:38:24 : 03/03  12.5%
11:38:29 : 04/04  6.3%
11:38:34 : 05/05  3.1%
11:38:36 : 06/06  1.6%
11:38:40 : 07/07  0.8%
11:38:42 : 08/08  0.4%
11:38:45 : 09/09  0.2%
11:38:48 : 10/10  0.1%
11:39:55 : Test finished.

----------
Total: 10/10 (0.1%)


Emese:  Acidic noise  / distorsion


foo_abx 1.3.4 report
foobar2000 v1.0.3
2013/02/26 12:49:54

File A: C:\stuff\music\abx\emese.wav
File B: C:\stuff\music\emese.mp3

12:49:54 : Test started.
12:50:04 : 01/01  50.0%
12:50:06 : 02/02  25.0%
12:50:08 : 03/03  12.5%
12:50:10 : 04/04  6.3%
12:50:12 : 05/05  3.1%
12:50:13 : 06/06  1.6%
12:50:15 : 07/07  0.8%
12:50:16 : 08/08  0.4%
12:50:18 : 09/09  0.2%
12:50:20 : 10/10  0.1%
12:50:21 : Test finished.

----------
Total: 10/10 (0.1%)
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-26 06:36:12
Looks like emese is easier for you to ABX than florida_seq.
Can you say something about the difference 3100i -V0+ vs. 3.100alpha2 -V0 and CBR320?
Title: lame3100h, a functional extension
Post by: shadowking on 2013-02-26 11:22:31
I used 3100i -b320 :

Florida - To me CBR seems worse. There is more smearing .

foo_abx 1.3.4 report
foobar2000 v1.0.3
2013/02/26 22:18:44

File A: C:\stuff\music\abx\florida_seq.wav
File B: C:\stuff\music\florida_seq.mp3

22:18:44 : Test started.
22:19:00 : 00/01  100.0%
22:19:10 : 01/02  75.0%
22:19:20 : 02/03  50.0%
22:19:23 : 03/04  31.3%
22:19:25 : 04/05  18.8%
22:19:27 : 05/06  10.9%
22:19:41 : 06/07  6.3%
22:19:44 : 07/08  3.5%
22:19:48 : 08/09  2.0%
22:19:51 : 09/10  1.1%
22:19:53 : Test finished.

----------
Total: 9/10 (1.1%)


Emese: Same or maybe worse than VBR V0+.


foo_abx 1.3.4 report
foobar2000 v1.0.3
2013/02/26 22:16:03

File A: C:\stuff\music\abx\emese.wav
File B: C:\stuff\music\emese.mp3

22:16:03 : Test started.
22:16:24 : 01/01  50.0%
22:16:30 : 02/02  25.0%
22:16:31 : 03/03  12.5%
22:16:33 : 04/04  6.3%
22:16:35 : 05/05  3.1%
22:16:37 : 06/06  1.6%
22:16:39 : 07/07  0.8%
22:16:41 : 08/08  0.4%
22:16:42 : 09/09  0.2%
22:16:50 : 10/10  0.1%
22:16:52 : Test finished.

----------
Total: 10/10 (0.1%)


Florida 3100i standard -V0

foo_abx 1.3.4 report
foobar2000 v1.0.3
2013/02/26 22:38:33

File A: C:\stuff\music\abx\florida_seq.wav
File B: C:\stuff\music\florida_seq.mp3

22:38:33 : Test started.
22:38:49 : 01/01  50.0%
22:38:51 : 02/02  25.0%
22:39:00 : 03/03  12.5%
22:39:02 : 04/04  6.3%
22:39:04 : 05/05  3.1%
22:39:10 : 06/06  1.6%
22:39:12 : 07/07  0.8%
22:39:16 : 08/08  0.4%
22:39:18 : 09/09  0.2%
22:39:20 : 10/10  0.1%
22:39:24 : Test finished.

----------
Total: 10/10 (0.1%)


I think -V0+ is a bit harder to abx than -V0, but they are similar.

CBR 320 brute force approach cannot be counted on - at least for now. Its also fat , encodes silence @ 320, mono @ x2 the data rate. Its the most popular setting on P2P filesharing.  Maybe LAME at its current state has CBR issue.. But no developer since v3.90 has ever considered cbr320 or cbr in general it as a solution to anything. For some rare legacy device support it does ok @ 192..224 while still decent size for portables . In some rare cases it can outperform VBR but you cannot count on that as the norm at all.
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-26 13:18:51
Thank you for testing.

I 'e got a file 'A03_emese.flac' on my pc which contains a 6 sec. track, and I guess that's your emese sample.
I've encoded it, and it's clear that the '+' variant can't help here. It consists of short blocks to nearly 100 per cent, so that bit reservoir is nearly always close to empty. So the '+' variants can't provide extra bits for short blocks, and the data space available from 320 kbps frames isn't sufficient here. Quality isn't real bad though for my taste, but I must admit I can't concentrate well on such 'music' - and I'm not sensitive to pre-echo  (though I've 'improved' a bit).
While searching for the sample in the www, I've seen it's such a hard sample that even AAC at still higher bitrates can't get it right.

I don't have 'florida_seq' on my pc, and a www search couldn't find me it (the ff123net samples page is not available any more). Would you mind providing this sample?
Title: lame3100h, a functional extension
Post by: shadowking on 2013-02-26 13:40:53
I've uploaded the sample here:

http://www.hydrogenaudio.org/forums/index....showtopic=99678 (http://www.hydrogenaudio.org/forums/index.php?showtopic=99678)
Title: lame3100h, a functional extension
Post by: BFG on 2013-02-26 14:44:39
Thank you for testing.

I 'e got a file 'A03_emese.flac' on my pc which contains a 6 sec. track, and I guess that's your emese sample.
I've encoded it, and it's clear that the '+' variant can't help here. It consists of short blocks to nearly 100 per cent, so that bit reservoir is nearly always close to empty. So the '+' variants can't provide extra bits for short blocks, and the data space available from 320 kbps frames isn't sufficient here.

Out of curiosity, how does 3.100 handle this sample at 640kbps freeformat?  (I'd check it myself but my ears are bad enough I can't hear most of the problems you're describing.)
Title: lame3100h, a functional extension
Post by: halb27 on 2013-02-26 15:35:06
My ears probably aren't better. I am able to ABX emese and florida_seq at low bitrate, but not at -V0 nor -V0+.

As for florida_seq:

I guess it's about frame #188 ... #192, which contain all short block granules except for the last one. Bitreservoir is high at the time of frame #188 (~ 4000 bit) for both -V0 and -V0+. So no benefit again in this situation for the bitreservoir saving strategy of the functional extension. When using -V0+ frame #188 is encoded at full -V0 accuracy (even a bit better according to the high default --adbr_short value of 510 kbps), but this is done at the expense of a nearly empty bitreservoir after encoding frame #188. Standard Lame behaves different. It's strategy doesn't allow to use more than ~10000 bit for a frame. This means that frame #188 is encoded with a lower accuracy than the internal quality requirements demand for (bits used are ~20 per cent below that of -V0+), but it keeps bitreservoir at ~2000 bit which is in favor of the following frames. Due to lacking data space frames #189 to #192 are all encoded below Lame's internal accuracy demands no matter -V0 or -V0+, but because of the bigger bitreservoir in the case of standard Lame -V0 uses ~10 per cent more bits on frame #189, and ~5 per cent more bits on frame #190. Bits used for frame #191 and 192 are roughly the same for -V0 and -V0+.

Because of the different bit allocation strategies -V0 can sound better than -V0+ on occasion. I prefer the -V0+ strategy because it is always better in the case of a sequence of only one or two short block granules (quite common with 'natural' music). Also in longer short block granule sequences I think it is better to throw all the bits needed at the first and second short block granules and accepting a somewhat lower accuracy for the third short block granule and also for the fourth. Just a belief though.
Lame's standard strategy can be approximated a bit by using -V0.5+ or similar together with --adbr_short 400 or similar because bits used are restricted this way for the short block granules thus preserving more bits from the bitreservoir for the following frames. Limiting the accuracy increase of 3100i by using --snrincmax_short 3.0 or similar for the highest quality settings can help as well to approximate the standard behavior.

Maybe it's a good idea to lower the default value of 510 kbps for short block granules, and/or use --snrincmax_short in a restrictive way. Has somebody a sample where a lower --adbr_short value and/or a small --snrincmax_short value results in an improved quality?