Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Resurrecting/Preserving the Helix MP3 encoder (Read 43889 times) previous topic - next topic
maikmerten, KevinB52379, Bogozo and 7 Guests are viewing this topic.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #25 –
It was actually "_M_IX86;IEEE_FLOAT" that were missing. I agree with all that you say, and without a 'lametag' equivalent, the problem will remain. I also seem to remember that lame had a frame analyzer but I'm not sure whether it compiles these days. I'll look into this further over the next few days.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #26 –
Cool, thanks! I'm looking into the Lame info tag specification and it's quite clear where it would have to be implemented. Looks pretty doable, but is of course a "once time permits" thing.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #28 –
Quote from: john33 on
Quote from: Porcus on
Quote from: john33 on
both decode to fewer wave samples than in the original wave file on the file tested
Dumb question, but have you checked the actual wave files, not just fb2k's user output?
There have been instances where fb2k versions have reported wrong sample counts.

No, not a dumb question at all. I hadn't appreciated that fb2k had that issue.

I should have been more precise, but I didn't recall precisely the findings - but now I do, as the issue was noted, along with a fix, in the v2 beta 13 changelog.

The issue was detected in the binary compare "component" (in v2 it isn't anymore a separate "component") and then only when comparing pairs with at least one file malformed one way or another. I think it even was WAVE files. I have no idea if it would even manifest itself outside binary compare.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #29 –
Noted, thanks. :)

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #30 –
Hi John, thanks for providing a working build for mp3x. It somewhat works on Linux with Wine (I cannot click anything - but I can operate it with the keyboard. Some weird issue with super-old GTK on Wine, I guess).

I could confirm that there sometimes actually are too few frames being emitted - it's not decoder trimming away samples. The encoding application works under the assumption that every call to the encoder will emit one frame of audio, but this isn't actually true: Very small frames seem not be emitted, they're only flushed out with additional audio frames being submitted. The lower the VBR quality, the fewer frames actually are being written out.

I now added logic to shove additional silent audio into the encoder until the expected number of frames is emitted and written to file. Now I'm getting a) the complete audio and b) consistent padding at the start and the end, no matter if CBR or VBR and no matter the VBR quality.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #31 –
Great work. As I was using the '-V150' option, that may explain why I didn't perceive that problem at the same level.

I'll download and recompile your modified code, then re-post it here for testing.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #33 –
Is it possible to leave tool recognition tag, like lame does when you check which tool encoded audio? It doesn't leave anything that foobar2000 recognizes.
Error 404; signature server not available.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #34 –
Quote from: itisljar on
Is it possible to leave tool recognition tag, like lame does when you check which tool encoded audio? It doesn't leave anything that foobar2000 recognizes.

Yeah, I'm currently reading into the LAME info tag specification with plans to implement that. It's needed for gapless playback anyways. No promises on when this will be ready, though, as I will have to implement this as time is available.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #35 –
I tested latest build and found a bug. When I convert mp3 from stdin with a dash in foobar2000 there is a truncation at beginning of the sample. With a wav file it's fine.

One more thing, when I published a post with "Helix MP3 Encoder 5.1.1" then in tomp3.cpp I changed the values from 16 to 4 to shorten the gap at the end of the tracks.
It's here and below in parentheses:
https://github.com/maikmerten/hmp3/blob/6edb64282e29b92029c3723bfbdbc913b43da301/hmp3/src/test/tomp3.cpp#L644
Now that there are no such problems with different lengths of gap, these lines of code are unnecessary and cause a longer gap. What do you think?

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #36 –
Hi KrajzegaX, I'll happily check out what reverting the 4 back to 16 will accomplish, thanks for the hint (I assumed those values were all original RealNetworks).

As for stdin-input: Thanks for testing. This might be a platform-specific thing, on Linux things seem to behave properly:

Code: [Select]
hmp3 - - < sine441-notches.wav > test.mp3

gives me a non-truncated output file, as enclosed screenshot illustrates. However, it's certainly good to know of that defect. That whole input code looks somewhat wonky to me anyways and I don't trust it in particular.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #37 –
Changing 4 to 16 in https://github.com/maikmerten/hmp3/blob/6edb64282e29b92029c3723bfbdbc913b43da301/hmp3/src/test/tomp3.cpp#L644 doesn't appear to have an effect on the gaps. (See enclosed image)

I consider this a good thing, as it may indicate that the new frame flushing logic works irrespective of how the input chunks were processed.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #38 –
Okay, got some initial LAME info tag going on there.

Code: [Select]
maik@maik-desktop:/tmp$ eyeD3 -P lameinfo test.mp3
/tmp/test.mp3                                                                                                                   [ 4.83 KB ]
--------------------------------------------------------------------------------------------------------------------------------------------
Encoder Version     : LAME-H511
LAME Tag Revision   : 2
VBR Method          : Unknown
Lowpass Filter      : 0
Encoding Flags      : 
ATH Type            : 0
Bitrate (Constant)  : 0
Encoder Delay       : 1675 samples
Encoder Padding     : 2304 samples
Noise Shaping       : 0
Stereo Mode         : Undefined
Unwise Settings     : False
Sample Frequency    : 48 kHz
MP3 Gain            : 0 (+0.0 dB)
Preset              : Unknown
Surround Info       : None
Music Length        : 0.00 Bytes
Music CRC-16        : 0000
LAME Tag CRC-16     : 0000

The padding values are placeholders, but at least I'm getting the values back that I wrote into the headers. Audacity also reads things and crops away the padding. (see screenshot)

Now, I planned to write "HELIX.511" into the encoder version field, but then tools won't recognize the info tag. For now I'm going for "LAME-H511" for compatibility, but I wonder if there's a clever way to avoid using the "LAME"-prefix and still be recognized...

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #39 –
Quote from: maikmerten on
As for stdin-input: Thanks for testing. This might be a platform-specific thing, on Linux things seem to behave properly:

Code: [Select]
hmp3 - - < sine441-notches.wav > test.mp3
I tried this line on the console "cmd" and indeed it doesn't truncate at the beginning. Is there a bug in foobar2000? I don't know. ¯\_(ツ)_/¯
Quote from: maikmerten on
Changing 4 to 16 in https://github.com/maikmerten/hmp3/blob/6edb64282e29b92029c3723bfbdbc913b43da301/hmp3/src/test/tomp3.cpp#L644 doesn't appear to have an effect on the gaps. (See enclosed image)

I consider this a good thing, as it may indicate that the new frame flushing logic works irrespective of how the input chunks were processed.
Nice that it ignores that "PCM buffer" code. Good job.  :)

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #40 –
Okay, I have the LAME Info tag now implemented somewhat complete.

Code: [Select]
maik@maik-desktop:/tmp$ eyeD3 -P lameinfo lunarsanctum.mp3 
/tmp/lunarsanctum.mp3                                               [ 5.33 MB ]
--------------------------------------------------------------------------------
Encoder Version     : LAME-H511
LAME Tag Revision   : 2
VBR Method          : Unknown
Lowpass Filter      : 16500
Encoding Flags      : 
ATH Type            : 0
Bitrate (Constant)  : 0
Encoder Delay       : 1680 samples
Encoder Padding     : 2028 samples
Noise Shaping       : 0
Stereo Mode         : Undefined
Unwise Settings     : False
Sample Frequency    : 44.1 kHz
MP3 Gain            : 0 (+0.0 dB)
Preset              : Unknown
Surround Info       : None
Music Length        : 5.33 MB
Music CRC-16        : 0000
LAME Tag CRC-16     : 0000

According to my limited testing, MPEG-1 (32 kHz, 44.1 kHz, 48 kHz) files now have a sample-accurate length when decoded with  a decoder supporting the LAME info tag.

(For MPEG-2 I disabled the info tag for now. This needs more testing, due to a different frame size.)

THIS IS EXPERIMENTAL. Don't mass-encode for long-term storage. Should be fine for portable "throw away" use.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #41 –
For reasons I don't quite understand, lame decodes an hmp3 encoded file accurately, as you describe, but fb2k ignores the tag info and decodes to the same length as the mp3. Similarly, fb2k reports the mp3 file length including the delay/padding whereas a lame encoded file is reported as the same length as the wave file. Interestingly, the little lametag program reports that the encoded mp3 file tag info only includes the version.
Code: [Select]
F:\Test-lame>lame --decode 101.mp3
input:  101.mp3  (44.1 kHz, 2 channels, MPEG-1 Layer III)
output: 101.wav  (16 bit, Microsoft WAVE)
skipping initial 2209 samples (encoder+decoder delay)
skipping final 2243 samples (encoder padding-decoder delay)
Frame#  9784/9785    80 kbps   MS

F:\Test-lame>lametag 101.mp3
LameTag - Reads the LAME tag from an mp3 file
Copyright (c) 2005 phwip
Release 0.4.1, compiled 2005-09-09

F:\Test-lame\101.mp3

LAME tags were introduced with LAME version 3.90, but
this file was encoded by an earlier version of LAME and
the header only contains a version string.

Encoder string:     LAME
Version string:     -H511?ð


F:\Test-lame>lametag 10.mp3
LameTag - Reads the LAME tag from an mp3 file
Copyright (c) 2005 phwip
Release 0.4.1, compiled 2005-09-09

F:\Test-lame\10.mp3
Tag revision:       0
Encoder string:     LAME
Version string:     3.101
Quality:            93 (V0 and q7)
Encoding method:    vbr new / vbr mtrh
Lowpass:            22,100Hz
RG track peak:      <not stored>
RG track gain:      -6.6dB (determined automatically)
RG album gain:      <not stored>
nspsytune:          yes
nssafejoint:        yes
nogap continued:    no
nogap continuation: no
ATH type:           5
Bitrate:            minimal (-b) bitrate 32
Encoder delay:      576 samples
Padded at end:      1,572 samples
Noise shaping:      0
Stereo mode:        joint
Unwise settings:    no
Source sample freq: 44.1kHz
MP3Gain change:     <none>
Preset:             V0: preset extreme (fast mode)
Surround info:      none
Music length:       8,352,387 bytes
Music CRC:          7BBD
Actual Music CRC:   7BBD
Info tag CRC:       E8D6
Actual InfoTag CRC: E8D6


F:\Test-lame>
10.mp3 (lame encoded) and 101.mp3 (hmp3 encoded) are both encoded from 10.wav. On reflection, it looks like the lametag program also checks the 'Version String' and it may be that fb2k does the same and ignores the info if the version string is not greater than 3.90. But I am only guessing. ;)

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #42 –
Oh my, thanks for testing. Guess I'm running into some implementation-specific behavior there.

Possible source for problems:

  • The encoder name and version. I'm trying to make Helix encodes distinguishable from LAME encodes, but it might be that I need to "borrow" a LAME version string
  • I don't compute CRCs for the info tag or the music frames. Perhaps info tags with mismatching CRCs are ignored.

I guess I'll have to install FB2K in Wine and see what's going on. Is that nifty little "lametag" program available somewhere?
 

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #44 –
Quote from: maikmerten on
Oh my, thanks for testing. Guess I'm running into some implementation-specific behavior there.

Possible source for problems:

  • The encoder name and version. I'm trying to make Helix encodes distinguishable from LAME encodes, but it might be that I need to "borrow" a LAME version string
  • I don't compute CRCs for the info tag or the music frames. Perhaps info tags with mismatching CRCs are ignored.

I guess I'll have to install FB2K in Wine and see what's going on. Is that nifty little "lametag" program available somewhere?
I think you're right. I just changed the version string to 'LAME39111' and fb2k picked up the delay and padding and decoded it correctly. 'Lametag', however, fails to see the tag at all so it may not be as useful/more restrictive.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #45 –
'LAME5.1.1' works in fb2k but not lametag. So, I think it's safe to assume that lametag is more restrictive and perhaps less useful.

Edit: I spoke prematurely!! While fb2k sees the info correctly for the mp3, it ignores it on decode!!!

Edit 2: LAME still decodes this correctly. It would seem, therefore, that while LAME knows what it's doing with the lametag, other programs seem to be imposing restrictions that LAME doesn't. Irritating and slightly annoying!

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #46 –
Quote from: john33 on
It would seem, therefore, that while LAME knows what it's doing with the lametag, other programs seem to be imposing restrictions that LAME doesn't.

Sounds like something to file a bug report to those programs for.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #47 –
Okay, yeah, confusingly fb2k seems to read the padding info just right, but doesn't actually seem to take that information into account correctly.

Going for "LAMEH5.11" for now. The "." makes it look better parseable as version number. The "H" behind LAME denotes "Helix" (I really really don't want to steal an actual LAME version string). Also, the string now contains "MEH", which is neat.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #48 –
Actually, LAME works with 'HELIX5.11', and decodes correctly. However, fb2k recognises 'HELIX5.11' and quotes delay and padding numbers which differ from what LAME reports but sum to the same number. In Audacity, the original and decoded wavefiles seem to be sample accurate timewise. fb2k does not decode correctly.

Re: Resurrecting/Preserving the Helix MP3 encoder

Reply #49 –
Yeah, LAME still decodes fine with HELIX5.11. However, my version of Audacity (2.4.2) doesn't seem to want to read the info tag if the encoder string doesn't start with LAME...

As for fb2k, I guess I might just ask for guidance in the fb2k forum.

And all of this started because I want to encode MP3 fallbacks for voice content on the web. ;-)