HydrogenAudio

Lossy Audio Compression => MP3 => MP3 - Tech => Topic started by: halb27 on 2013-08-20 13:57:56

Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-20 13:57:56
What’s lame3100l?
It's an extension to Lame 3.100 alpha2 which offers constraint vbr mode.
-Vn defaults to this constraint vbr mode. Average bitrate increase for quality levels -V5 to -V0 is only 2 kbps (for a variety of pop music) compared to original Lame.
Audible advantage of this variant is rare. It can be experienced for instance with samples eig (http://dl.getdropbox.com/u/2681777/Lame3100k/eig.flac) and lead-voice (http://dl.getdropbox.com/u/2681777/Lame3100k/lead-voice.flac).
Using a very high quality -Vn level and a very demanding constraint mode option (see below) allows for a better signal to noise ratio than when using -V0. This can help bring very hard problems like harp40_1 (http://dl.getdropbox.com/u/2681777/Lame3100k/harp40_1.flac) a bit further to perfection.

--cvbr x, 9 >= x >= 0, or x = -1
Constrained vbr mode is controlled by cvbr level.
Lowest cvbr level is 9.0, highest cvbr level is 0.0.
Default cvbr level for -V5 (and lower) is 9.0, -V4 defaults to cvbr level 8.0, ..., and -V0's default cvbr level is 4.0.
Using the --cvbr x option overrides the default cvbr level.
Using a cvbr level better than 4.0 together with -V0 or -V1 can improve those small issues which still exist when using -V0. Try samples herding_calls (http://dl.getdropbox.com/u/2681777/Lame3100l/herding_calls.flac) and  harp40_1 (http://dl.getdropbox.com/u/2681777/Lame3100k/harp40_1.flac) for a corresponding listening experience.
Interesting example settings:
-V0 --cvbr 0 (yielding 316 kbps on average) is an extremely high quality alternative to cbr 320.
-V0.5 --cvbr 3.5 (266 kbps) is a bit saving very high quality alternative to this.
--cvbr -1 turns cvbr mode off and falls back to original Lame behavior.

--bCVBR x, 135 <= x  <= 316
x is the desired average bitrate for a variety of pop music.
For x <= 230 or x >= 278 -Vn is chosen internally together with the corresponding default cvbr level in such a way that average bitrate is x. For 230 < x < 278 however there is a certain bias towards a somewhat higher cvbr level. This has a positive effect on the small remaining issues mentioned before with harp40_1 and herding_calls.

For more details, especially on installation, see the provided documentation.
lame3100l can be downloaded from here (http://dl.getdropbox.com/u/2681777/Lame3100l/lame3100l.zip).
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-20 14:11:37
As for the stopped discussion in the 3100k thread:

When somebody wants to publish a variant to my variant I'd like him to contact me. We should try to find a solution which makes its way into a new version. Defaulting to another lowpass is not interesting however as we have the --lowpass option, so everybody can use a lowpass according to what he prefers.

Sure everybody can create a personal version for personal use, but please don't publish it. I think nobody wants a forest of variants.
This holds for my variant as well, and I'd welcome very much if my modification would make it into official Lame, so I could stop creating my variant.

Title: lame3100l - a constraint vbr variant
Post by: halo001 on 2013-08-27 05:47:54
So it is possible that your variant will officially be part of Lame Project? If Yes, I'll be very glad to hear that especially with the current development of your work.
but I have a question about as of usage, can I use lame3100l with foobar to encode my tracks directly from FLAC?
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-27 06:49:30
...can I use lame3100l with foobar to encode my tracks directly from FLAC?

Sure you can, using foobar's command line interface for encoding.
Right-click on a FLAC track of your playlist, choose 'Convert', '...', click on 'Output format', click on 'Add New', and describe your encoder like this:

(http://dl.getdropbox.com/u/2681777/Lame3100l/foobar.gif)

Modify my Encoder file path and personal setting --bCVBR 266 to what you use.
You need to do this only once. Later just select your new encoder when you want to encode.
Title: lame3100l - a constraint vbr variant
Post by: halo001 on 2013-08-27 08:24:03
(https://wgzsjw.bn1.livefilestore.com/y2ppUkBXZsvHAsV6Y5tiQgGaI_kXdpjMA9TnsBoelzuMHXKSdWIMvVtVD8wWPsi2YMvq_EtFVUbWAMP0jDUuS5Ho5V3xtimwzV1sgYt2PW7dhg/Screenshot.png)


Thank you very much. Works fine except for one thing. I noticed that there are file size variation between track encoded directly to lame3100l.exe (via cmd) as track encoded on foobar (via foobar's commandline interface). I encoded a single track, 3:55 in Length both with the same settings but the file size differ from one another. and these are the resulting file size:

1. 9,053 KB - Encoded via cmd (1); and
2. 9,126 KB - Encoded using foobar (2)

(Note that I use the setting on the image)


1. 6,023 KB - (1)
2. 6,097 KB - (2)

and this time I encoded the track with settings -V2 --cvbr 5.5 -ms --replaygain-accurate --lowpass 19205

The filesize is somewhat larger with foobar than cmd.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-27 11:18:03
I guess foobar writes tags to the file after encoding.
Title: lame3100l - a constraint vbr variant
Post by: Kamedo2 on 2013-08-30 03:50:53
Something is strange. My CPU is sometimes in idle during the execution. FFmpeg never had such behavior.
I'm running 2 lame3100ls simultaneously on the Intel Core i7 870, and it may be causing the strange behavior.
Title: lame3100l - a constraint vbr variant
Post by: Kamedo2 on 2013-08-30 04:32:43
Apparently, this encoder is about 5x slow when the file name includes two-byte characters. It also happens when only one lame3100l was executed per time.
It can be lame3100l side problem, or mp3packer side problem, or some bugs in my script. Windows 7 Home Premium 64bit was used.

I'll figure it out, but it'll take time.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-30 08:26:09
Can you give an example of such a file name?
Title: lame3100l - a constraint vbr variant
Post by: Hex144 on 2013-08-30 08:46:28
Something is strange. My CPU is sometimes in idle during the execution.


I encounter a similar problem quite frequently when encoding via foobar and with mp3Packer.
The packed file cannot be overwritten onto the original mp3 and the encoding speed in the foobar converter drops to zero for ~30 sec, then I end up with two files: the original lame output and another suffixed with "_mp3packer".
Of course, I have to reencode it, because the packed file isn't tagged.
It may be a Windows "file-in-use"-type glitch.
A bit of a PITA.

PS: I haven't paid close attention but iirc, it happened quite randomly, not necessarily with unicode file names.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-30 11:06:32
Now that you write this I remember having had similar issues on occasion. It happened very rarely though and I also thought of timing problems.
Title: lame3100l - a constraint vbr variant
Post by: Kamedo2 on 2013-08-30 15:49:48
I tested using 13 diverse songs at --bCVBR 128kbps, 144kbps, 160kbps, ... , 304kbps, 320kbps(13 bitrates).

lame3100l without mp3packer : 14.3x ~ 25.8x realtime
lame3100l with mp3packer without two byte characters : 13.4x ~ 25.5x realtime
lame3100l with mp3packer with two byte characters : 1.3x ~ 3.4x realtime

As we can see, this encoder is reliably slow when the mp3packer is in the same folder and the wav file name includes two byte characters.
And this encoder is particularly slow(1.3x) when the wav file is short, and when that happens, the CPU usage is mostly less than 2%.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-30 17:22:49
Please give me an example for such a filename so I can have a look into the issue.
Title: lame3100l - a constraint vbr variant
Post by: Kamedo2 on 2013-08-31 00:59:45
Example of a filename that makes this encoder pause.

11. Полюшко-поле ''Le Vent Vert''_short.wav
Code: [Select]
ABC-HR>lame3100l.exe --bCVBR 256 "11. Полюшко-поле ''Le Vent Vert''_short.wav" 
"11. Полюшко-поле ''Le Vent Vert''_short.mp3"
LAME 3.100 (alpha 2, Jul 24 2013 18:42:25) extension:l 32bits (http://lame.sf.ne
t)
warning: alpha versions should be used for testing only
Using polyphase lowpass filter, transition band: 17249 Hz - 17782 Hz
Encoding ABC-HR\11. Полюшко-поле ''Le Vent Vert''_short.wav
      to ABC-HR\11. Полюшко-поле ''Le Vent Vert''_short.mp3
Encoding as 44.1 kHz j-stereo MPEG-1 Layer III VBR(q=0.647059)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  115/115  (100%)|    0:00/    0:00|    0:00/    0:00|  15.895x|    0:00
 32 [  0]
 40 [  0]
 48 [  0]
 56 [  0]
 64 [  0]
 80 [  0]
 96 [  0]
112 [  1] %
128 [  5] %%%%
160 [  0]
192 [  0]
224 [  3] ***
256 [ 94] %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
320 [ 12] %%%%%%%%*
-------------------------------------------------------------------------------
  kbps        LR    MS  %    long switch short %
  255.0      94.8  5.2        94.3  3.5  2.2
Writing LAME Tag...done
ReplayGain: +6.7dB

It stops after printing ReplayGain: +6.7dB, and after about one minute, the cmd restarts.
And the unicode mp3 seems to be unpacked and it's compressible by the mp3packer from the cmd (no stops in here).
Title: lame3100l - a constraint vbr variant
Post by: BFG on 2013-08-31 03:39:26
Now that you write this I remember having had similar issues on occasion. It happened very rarely though and I also thought of timing problems.

I've had this happen on occasion too.  It at least occasionally appears to be due to Windows trying to access the album art within one of the files in order to create a thumbnail for it.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-31 14:03:54
Example of a filename that makes this encoder pause.

11. ???????-???? ''Le Vent Vert''_short.wav
...

I renamed a short wav file to your name using Explorer and copy & paste for the filename. No problem.
I tried to use your command line using cmd, but I couldn't get your filename in. Copy & paste doesn't work here.
However I could use foobar for converting this file using your command line. No issue whatsoever, also mp3packer worked fine.
I'm on a Windows 7 Pro SP1 64 bit system and use foobar2000 v1.2.9.

Have you tried using foobar?
Title: lame3100l - a constraint vbr variant
Post by: [JAZ] on 2013-08-31 15:37:02
You don't need to write the whole name in windows vista/7 as the cmd client is better than in XP and does autocomplete with the tab key. Start writing the name (in this case 11.) and press the tab key. It should autocomplete it.

It can even autocomplete with wildcards, i.e writing  *.wav and pressing tab several times will loop between all wave files under that directory

Btw, the reason why they don't show properly in the console is because the default font doesn't handle unicode. Right click on the window icon (the one in the titlebar), properties, font and there select "Consolas" or "Lucida console" instead of bitmapped font.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-31 16:22:46
Thank you, JAZ, changing the font from within cmd did the trick. I could confirm the problem.

The problem is that mp3packer isn't used, but it looks like it's tried to be used intensively until it's all stopped. And obviously it's the parameter passing to mp3packer which doesn't work in this case.
Strange though that everything is fine when used from within foobar.
Well, I can look into the code whether I can fix it, but, Kamedo2: do you really encode from the command line for practical purposes? For mere looking at lame3100l's behavior because of Lame's display output it's not necessary to use mp3packer.
Title: lame3100l - a constraint vbr variant
Post by: Kamedo2 on 2013-08-31 17:21:35
do you really encode from the command line for practical purposes?


No, practically, I use LAME via Exact Audio Copy(EAC). And I use WSH+Javascript to test the validity of codecs. If anything goes wrong, I use cmd to see whether it's a bug in my script, or it's the encoder side problem.

I already encoded hundreds of mp3s by my script while I was sleeping, and I've confirmed that lame3100l+mp3packer is always slow with unicode filename, always fast with one-byte filename.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-31 20:06:44
I've looked up the code. I've found a spot where my treatment of character strings probably isn't adequate for unicode strings. It explains why a lot of time is spent without significant cpu usage. I'll try to fix it, but I'm afraid it won't help much because I think mp3packer isn't called in a way that works (because otherwise a <filename>_mp3packed.mp3 file would exist). I'll look into it.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-08-31 21:16:41
OK, looks like the real problem is that I call mp3packer using ShellExecuteA. I should use ShellExecuteW in the case of unicode filenames.
I'll fix that within the next days.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-09-01 14:11:16
I just realized that current mp3packer does not work with your filename (though it is said to support Unicode).
So I will not change lame3100l.

A workaround in your script can be what foobar does: rename the file as the first operation, and re-rename it as the last.
Title: lame3100l - a constraint vbr variant
Post by: [JAZ] on 2013-09-01 14:56:04
I beg to differ:
Code: [Select]
E:\programes\AudioCodecs\lamehalb>mp3packer.exe "11. Полюшко-поле ''Le Vent Vert''_short.wav.mp3"

*** '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav.mp3' -> '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav-vbr.mp3'
100% done with 542 frames (60.51x)

E:\programes\AudioCodecs\lamehalb>ls -l
total 6640
----------+ 1 JosepMa None  412906 Sep  1 15:46 11. Полюшко-поле ''Le Vent Vert''_short.wav-vbr.mp3
----------+ 1 JosepMa None  431386 Aug 31 17:12 11. Полюшко-поле ''Le Vent Vert''_short.wav.mp3



E:\programes\AudioCodecs\lamehalb>mp3packer.exe -u "11. Полюшко-поле ''Le Vent Vert''_short.wav.mp3"

*** '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav.mp3' (backup to '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav-vbr.mp3')
Do you really want to overwrite '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav-vbr.mp3'? (y/n)
y
100% done with 542 frames (113.27x)



E:\programes\AudioCodecs\lamehalb>mp3packer.exe --copy-time "11. Полюшко-поле ''Le Vent Vert''_short.wav.mp3"

*** '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav.mp3' -> '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav-vbr.mp3'
100% done with 542 frames (456.72x)

E:\programes\AudioCodecs\lamehalb>dir
 El volumen de la unidad E es DATA
 El número de serie del volumen es: C024-FDFC

 Directorio de E:\programes\AudioCodecs\lamehalb

01/09/2013  15:51    <DIR>          .
01/09/2013  15:51    <DIR>          ..
31/08/2013  17:12          412.906 11. Полюшко-поле ''Le Vent Vert''_short.wav-vbr.mp3
31/08/2013  17:12          431.386 11. Полюшко-поле ''Le Vent Vert''_short.wav.mp3




E:\programes\AudioCodecs\lamehalb>mp3packer -z --copy-time -u --process sse41 "11. Полюшко-поле ''Le Vent Vert''_short.wav.mp3"
WARNING: SSE4.1 unsupported

*** '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav.mp3' (backup to '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav-vbr.mp3')
Do you really want to overwrite '11. ðƒð¥ð╗ÐÄÐêð║ð¥-ð┐ð¥ð╗ðÁ ''Le Vent Vert''_short.wav-vbr.mp3'? (y/n)
y
100% done with 542 frames (25.19x)

E:\programes\AudioCodecs\lamehalb>dir
 El volumen de la unidad E es DATA
 El número de serie del volumen es: C024-FDFC

 Directorio de E:\programes\AudioCodecs\lamehalb

01/09/2013  15:53    <DIR>          .
01/09/2013  15:53    <DIR>          ..
31/08/2013  17:12          431.386 11. Полюшко-поле ''Le Vent Vert''_short.wav-vbr.mp3
31/08/2013  17:12          412.904 11. Полюшко-поле ''Le Vent Vert''_short.wav.mp3


Its display does not support unicode, but filenames are properly used in all cases. Version 2.04
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-09-01 22:35:24
You're right, sorry.
I had got the error message 'No Input given', but I must have done something wrong.
I just tried again, and everything was fine.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-09-02 18:11:18
I have fixed the issue.
When downloading again it should work now.
Title: lame3100l - a constraint vbr variant
Post by: Kamedo2 on 2013-09-02 18:28:18
I tried 30 diverse, both double-byte and single-byte file name wav files and it seems to be working. It's running more than 20x realtime on my Core i7 870.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-09-03 12:32:15
I had to do another change in order to fix this:
a) With my last change I forgot to free up the memory allocated to take up the Unicode filenames.
b) I noticed that when invoking lame3100l from the cmd prompt, it was necessary to qualify the full lame3100l path in order to make use of mp3packer.

Please download again. Sorry.
Title: lame3100l - a constraint vbr variant
Post by: Kamedo2 on 2013-09-03 14:33:50
I noticed that this encoder is rather slow on speech or silent content. I remember that the quality is perceptually worse in SSE version, so they can be something to do with denormals.
Title: lame3100l - a constraint vbr variant
Post by: IgorC on 2013-09-20 02:46:16
We were discussing about how MP3 handles sharp transients in electronic music
MP3 320 kbs vs Redbook ripped to WAV (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=102679&view=findpost&p=845298) topic.

Here is a result for MP3 (~300 kbps) and AAC (~192 kbps) on EIG sample (electronic music with a lot of transients)

MP3 encoder > LAME 3.100l V0
AAC encoder > Apple, QAAC tvbr q90

files: here (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=102079&view=findpost&p=845333)

Log:
Code: [Select]
ABC/HR Version 1.1 beta 2, 18 June 2004
Testname:

1L = E:\Audio\EIG\eig  LAME 3.100l V0.wav
2R = E:\Audio\EIG\eig Apple TVBR q90.wav

---------------------------------------
General Comments: A 2º sample is transparent

---------------------------------------
1L File: E:\Audio\EIG\eig  LAME 3.100l V0.wav
1L Rating: 3.8
1L Comment:
---------------------------------------
ABX Results:
Original vs E:\Audio\EIG\eig  LAME 3.100l V0.wav
    5 out of 5, pval = 0.031

MP3 isn't enough good while AAC is transparent.

Edit: As usual, MP3 suffers from very audible pre-echos.
Title: lame3100l - a constraint vbr variant
Post by: GeSomeone on 2013-10-27 13:54:44
I noticed the the lowpass in 3100L was changed back from 18200 to 17500 (as it was before 3100g). As I did not remember this mentioned anywhere, I hereby do. It might even happened accidentally (or I missed something).
BTW I found this at -V 1 and I know it isn't a problem but I think you should be aware of it.
Title: lame3100l - a constraint vbr variant
Post by: halb27 on 2013-10-28 17:35:57
Yes, lowpass is 17500 now for -V1, it's 18200 for a -V level better than 0.5.