Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Monkey's Audio Adds 32-bit Float Support (Read 6803 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Monkey's Audio Adds 32-bit Float Support

Hi everyone,

I just wanted to announce that Monkey's Audio now supports 32-bit floating point files with the beta download:
https://www.monkeysaudio.com/download.html

David Bryant (of WavPack) told me it would take months.  Then Robert Kausch delivered working code!  It's not quite as good as WavPack, but I'm just amazed it's working!

Testing and feedback appreciated.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #1
Let the Monkey flow.

Thanks!

Re: Monkey's Audio Adds 32-bit Float Support

Reply #2
Shouldn't that be "float", not "flow"?  ;)


@MonkeysAudio , since you are filling in new features, and since I'm no stranger to pestering devs with all sorts of ideas even when I'm using it only for testing ... :

* CLI help: "-r" means remove all tags?
* The GUI can report to the user which compression level was used.  The CLI?  Giving MAC.exe , it returns no info about that.  There probably is one such option, maybe document it? 
* And in the GUI ... wouldn't be the worst to have a column for it and a button to scan for info.  Actually, it could be reported when verifying too. "OK (High)". Same about source / what it will be decoded to:  "OK (from AIFF @ High)".
* The CLI output help suggests "   Transcode from pipe: ffmpeg.exe -i" [etc.]
Problem is, that is not lossless if the input file is > 16 bits.  The evil ways of ffmpeg: "-f wav" delivers 16 bits, truncated. Most destructive, float above 0 dBFS will be clipped.  Not sure if it is a good idea for a lossless compressor to solicit a command-line that suddenly is lossy. Blame ffmpeg! https://hydrogenaud.io/index.php/topic,123585

Now as for encoding & related - now that Monkey's supports more inputs than just WAVE file:
 
* Is there any other way to nuke WAVE (/AIFF) metadata rather than piping in?  (It seems to me that full file restore was mandatory until the pipe support?)
It doesn't appear that hard to say, "-k" to kill metadata, leaving in a fresh minimal WAVE header or WAVE_FORMAT_EXTENSIBLE if it is multi-channel with a channel mask I guess.  Or - should any user ever want to set it that way - "-k AIFF" to give it a minimal AIFF header instead. Not sure if that will be the most used feature ...)

Which brings me to, here is just an idea: 
* Monkey's doesn't support "wasted bits", but it could detect "fake 24-bits" if the full file is really 16 - and, subject to user demand, encode as 16.  That isn't wise if the user wants full file restore, but if user doesn't want that, there could be a capital "-K" for such. Of course that would have to alter the original file headers, so it would imply the suggested "-k":
MAC.exe infile.aiff outfile.ape -c3000 -K will not only kill all the AIFF metadata, but also detect any "fake 24 bits file" and encode as if it were 16. (And "fake 32" ... I kid you not: since ffmpeg does weird things about 24-bits, it sometimes pads them up to 32-bit integer, at least when converting to WavPack, so users who have fiddled around with ffmpeg on 24-bit files might get a 32 ... and if ffmpeg first has touched it, file headers are not worth preserving anyway.)
Well maybe one should for compatibility think twice before decimating "8 bits in fake 16", but anyway - all this is just "idea".

And for compression. 
* Every now and then there shows up a file which Monkey's will compress better in a lower mode. Especially high resolution: http://audiograaf.nl/losslesstest/revision%205/Average%20of%20all%20hi-res%20sources.pdf 
There is an "easy fix" that maintains all sorts of compatibility: compress more modes and keep the smallest file. Of course with the downside that the user reads "Extra High" and doesn't know whether "Insane" was ever attempted, so this is for users who know what they are doing. But I have a hunch that users who are willing to compress at "Insane", will happily want a smaller and lighter file.
You could consider for example options like:
-c3200: Compresses with "High", then starts over with a "Normal" (the "2").  Selects smallest file, discards the other.
-c5420: Compresses with "Insane" then starts over with Extra High" (the "4"). If that is bigger, stop. If that is smaller, run a "Normal" (the "2"). In the end, smallest is kept.
-c4444: Brute-force: do all four, now matter how stupid it sounds.  Hey -5555 will only double the encoding time.
Similar for re-encoding.  And maybe a simple "-n+" for "read the mode from the file, try encode one step up, keep smallest".
(I have no idea whether the Monkey's file format can accommodate switching between modes mid-stream ... and even if it does, whether the ffmpeg decoder would choke on it.)

Again, just loose swiney ideas. My familiarity with Monkey's is mainly through FOR looping for tests, so in case of "this is behind <menu>!" ... apologies.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #3
Uhmmm... does this means that can losslessy encode DSD (like WavPack already do) now ?
F.O.R.A.R.T. npo


Re: Monkey's Audio Adds 32-bit Float Support

Reply #5
WavPack's DSD support started in version 5 (2016) but 32-bit float has been supported for a very long time (2007 or earlier) so I don't see how the two things are related, except if APE developers want to actively compete with WavPack and others. However the competition would be more useful in other aspects (e.g. decoding speed). Even for a desktop user APE is still noticeably sluggish when doing stuff like RG scan and integrity check.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #6
If you are willing to step outside a player and use the application itself, then Monkey's and OptimFROG and WavPack 5 have faster integrity checking without decoding: https://hydrogenaud.io/index.php/topic,122094.msg1007783.html#msg1007783
(FLAC has been updated after that thread, and will now try to keep the length correct by muting a corrupted frame rather than skipping.)

Of course if you want to do AccurateRip retro-verification, you need the decoded audio.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #7
Not specific to this release but it would make much more sense to just block the decoding and show an error message instead of showing garbage or silence if the bundled APE.flt is not intended to support 32-bit decoding.
X

Re: Monkey's Audio Adds 32-bit Float Support

Reply #8
Good find that Cool Edit wasn't working with 32-bit float APE files yet.  I'll release an update at some point.  Cheers.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #9
Thanks. Currently the plugin always save as 24-bit but that's fine because I don't encode. However, externally encoded 32-bit files, both integer and float should be handled properly or completely disabled if 32-bit support is not planned.

I realize the version of Audition I have is lossy when handling externally encoded 32-bit integer files, but that's fine, just decode it in a usable way.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #10
Saving is a bit of a rat's nest.  Cool Edit only supports 32-bit data in a proprietary format.  So today I just convert that format to standard 24-bit integer.  I checked tonight and opening a 32-bit integer and 32-bit floating point file then saving provided the exact same calls.  It doesn't identify any difference.  I could probably switch to saving as 32-bit, just converted to the standard, but I don't think it's a real high priority.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #11
I think the difference is you are using Cool Edit but I am using Audition 1.5 and they are not exactly the same thing even though they share the same plugin. Since you mentioned WavPack I can report that WavPack's plugin handles all the supported 32-bit formats in Audition 1.5 without any issue. That said, I don't expect APE to achieve the same compatibility when saving (because if I save 32-bit files I would use WavPack), but for decoding, just in case when I get a 32-bit APE file from someone else, it should be handled correctly or completely disabled, instead of taking time to decode but ends up with something unusable.

Here is an excerpt from Audition 1.5's manual:
Quote
    32-bit Normalized Float (type 3) - Default is the internal format for Adobe Audition and the standard floating point format for type 3 .wav files. Values are normalized to the range of +/-1.0, and although values beyond this range are saved, clipping may occur in some programs that read them back in. (Adobe Audition won't clip audio but will instead read the same value back if it's beyond this range.)

    24-bit Packed Int (type 1, 24-bit) saves straight 24-bit integers so any data beyond the bounds is clipped. The .wav BitsPerSample is set to 24 and BlockAlign is set to 3 bytes per channel.

    24-bit Packed Int (type 1, 20-bit) saves straight 24-bit integers so any data beyond the bounds is clipped. The .wav BitsPerSample is set to 20 and BlockAlign is set to 3 bytes per channel. The extra 4 bits are actually the remaining valid bits when saving, and they are used when reading (thus still giving 24-bit accuracy if those bits were actually present when writing). Applications either fill those last 4 bits with zeros or with actual data; analog/digital converters that generate 20 bits of valid data automatically set the remaining 4 bits to zero. Any type 1 format with BlockAlign set to 3 bytes per channel is assumed to be packed integers, and a BitsPerSample value between 17 and 24 will read in all 24 bits and assume the remaining bits are either accurate or set to zero.

    32-bit 24.0 Float (type 1, 24-bit) - Non-Standard saves full 32-bit floats (ranging from +/-8million), but the .wav BitsPerSample is set to 24 while BlockAlign is still set to 4 bytes per channel.

    16.8 float - Obsolete/Compatibility is the internal format used by Adobe Audition 1.0. Floating point values range from +/-32768.0, but larger and smaller values are valid and aren't clipped since the floating point exponent is saved as well. The .wav BitsPerSample is set to 32 and BlockAlign is set to 4 bytes per channel.

    Enable Dithering dithers 32-bit files when they are saved to a PCM format (20-bit, 24-bit, or 32-bit). This option is available only for a 32-bit file that you select to save to a nonfloating-point type format. It applies a Triangular dither with a 1.0 depth 1.0 and no noise shaping. If you wish to apply a noise-shaped dither, use the Edit > Convert Sample Type command to dither the audio first, and then save the file without dithering enabled in the file format options.

The screenshot in the post below is from the older Cool Edit Pro 2 manual, notice this version uses 16.8 float as the internal format, but also allows saving in the standard format (0.24):
https://hydrogenaud.io/index.php/topic,114816.msg992983.html#msg992983

The even older Cool Edit Pro 1.x indeed only supports 16.8 float but not any other 32-bit format.


Re: Monkey's Audio Adds 32-bit Float Support

Reply #13
Both integer and float 32-bit files work as expected, excellent, thanks!

Re: Monkey's Audio Adds 32-bit Float Support

Reply #14
@MonkeysAudio : Bugs & more.

* Bug in Monkey's: Floating-point AIFC encodes to static. So does floating-point AU/SND (never saw one in the wild, created specifically to test using ffmpeg -codec copy; the .au plays well in foobar2000 at least).
The issue isn't solely endianness: foobar2000 bitcompares them to 770 dB difference.
(Is there a point in floating-point AU? It is a point to reject what you cannot process. Or maybe, if the decoding library offers the opportunity: forcing the player to read and honour the original file's header? Because Monkey's does restore the file perfectly.)

* Also some non-compliant float file which is also encoded to static:
I actually first discovered the previous bug by trying to ape the float file downloadable from https://soundcloud.com/kyrokotei/dream (you need to be logged in to Soundcloud to get it). I found that only because I was googling for 32-bit files at Soundcloud; those files put out there may reflect whatever potentially stupid software that is actually used to create such file.
And this file isn't compliant. Now, Monkey's accepts that file and encodes to static - but I don't know whether that is solely because of the previous item; even if that is fixed, the flaw in the file could make it harder to identify as float, so here you got a test case.

* Float WAVE that is rejected: Attached.
The "problem about that" is that it was created when youtube-dl by way of ffmpeg added metadata to a file. I got it from https://soundcloud.com/erikerixontechno/affenkafig-karnevalsrave-2018-bogen-2-koeln , but the file itself using the download link is perfectly ape-able (hey, it should be given the file name :-o) .
The one created when ffmpeg adds metadata is not. Rather than uploading a 2 GB file, I created an empty file at the same size, and pasted the beginning of the offending file into it. There are file headers, a tiny bit of audio and then all zeroes.


And finally, something that isn't lossless devs' fault at all, but lossless users' problem if they don't know, hence I think a bad recommendation:

* Pipe input, the suggestion in the mac.exe help and the bottom of https://www.monkeysaudio.com/help.html , of using ffmpeg with -f wav | mac.exe -
That ffmpeg command is lossy "except by coincidence" (and that coincidence materializes for 16-bits). But in the context of float, it is quite bad: if input exceeds plus/minus 1, ffmpeg will clip.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #15
Thanks for testing.

Do you have a sample for this one:
"Bug in Monkey's: Floating-point AIFC encodes to static. So does floating-point AU/SND"

The WavPack file issue was because I was skipping any file with a marker that wasn't PCM was getting rejected.  I added float for the next build.

Here's the history:
Fixed: WAV files with an extra marker would only encode if they were PCM (float wasn't passing the test).

Thanks again.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #16
I downloaded that file from Sound Cloud.  It was 2.17 GB.  I compressed it with Monkey's Audio and am playing it now in Media Center.

So what goes wrong?

Thanks.


Re: Monkey's Audio Adds 32-bit Float Support

Reply #18
@MonkeysAudio :
Attached a two files of one second. In fb2k, the 1secaiff.ape shows up as integer and the .au as float.
On second thought I have not the knowledge to rule out the error being in fb2k, but I assumed that it would do things right since it uses Monkey's 10.28.

Also, Monkey's can hardly reduce file sizes of floating-point .aif. Try the sample from Peter Kabal at McGill , it becomes larger than uncompressed. Convert it to 32-bit floating-point WAVE, and Monkey's compresses it by a third.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #19
1secau.ape decompresses to an AU file that's 32-bit.

Maybe try this test build and let me know how it's working:
https://www.monkeysaudio.com/files/MAC_1030_x64.exe

It has these two changes:
Fixed: WAV files with an extra marker would only encode if they were PCM (float wasn't passing the test).
Fixed: Floating point AIFF compression was not working properly.

Thanks.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #20
1secau.ape decompresses to an AU file that's 32-bit.
Yeah, decompressing to file using Monkey's would get me bit-exactly the same file back.
But playback ...

Maybe try this test build and let me know how it's working:
https://www.monkeysaudio.com/files/MAC_1030_x64.exe
Now fb2k tells me that the Monkey'ed floating-point AIFF is float. But, bitcompare:
File #1 peaks: 0.540265 (-5.35 dBFS) 0.551933 (-5.16 dBFS)
File #2 peaks: -9223372036854. (+770.60 dBFS) -9223372036854. (+770.58 dBFS)

Same goes for the AU I think. (Edit: Do the latter figures have sane sign, or has fb2k messed up a signed/unsigned declaration?)

Not good - but: is it supposed to work with a playback engine that uses 10.28?

Also, are any of these files supposed to return the same MD5 checksum? (They don't.)
I don't know what Monkey's will include in that; audio only, or stored headers too?

Re: Monkey's Audio Adds 32-bit Float Support

Reply #21
Please test JRiver Media Center because it uses the official SDK.

Foobar uses a colored version of the SDK so I'm never as sure what might be happening there.

From MACLib.h about the MD5:
Quote
MD5 Hash:

    Since the header is the last part written to an APE file, you must calculate the MD5 checksum out of order.
    So, you first calculate from the tail of the seek table to the end of the terminating data.
    Then, go back and do from the end of the descriptor to the tail of the seek table.
    You may wish to just cache the header data when starting and run it last, so you don't
    need to seek back in the I/O.

Re: Monkey's Audio Adds 32-bit Float Support

Reply #22
No problem for the 10.30-compressed file(s) in JRiver (32-bit, I had trial period left).

@Peter :
Now it seems to be fb2k related, mishandling big-endian float (AIFC or AU) compressed with Monkey's Audio. For the attached sample file (encoded with MAC.exe version 10.30) and the above attached 1secau.ape :
* Both play well in 32-bit JRiver,
* Botched up by fb2k (32-bit fb2k preview 2023-12-11)
* fb2k's bitcompare says they are equal, but with numbers that suggest some signed/unsigned issue:
Channel peaks: -9223372036854. (+770.60 dBFS) -9223372036854. (+770.58 dBFS)


Re: Monkey's Audio Adds 32-bit Float Support

Reply #24
foobar2000 seems to have fixed it in today's preview. Now handles .ape from big-endian float formats.
Since I tried to convert that second into quite a lot of formats, I discovered an issue with floating-point RF64 in both Monkey's and foobar2000; the issue could be in ffmpeg.

The attachment was created by ffmpeg -i .\1sec.w64 -codec copy -rf64 always 1sec.rf64.wav
* Monkey's doesn't compress it right. Yeah, tried JRiver (32-bit).
* foobar2000 gives it wrong length.

That suggests it is something awkward with the file even if WavPack handles it.
But more, both Monkey's and foobar2000 handle a WavPack-generated RF64. Since I cannot force wvunpack to generate small RF64 files, I am not attaching. But I generated a too big floating-point W64 and converted it to RF64 by wavpacking it and wvunpack -w, which outputs RF64 if and only if it is too big for WAVE. And generates its own headers.
Unfortunately I don't know any other way than ffmpeg to create RF64 out of a short file.

So they are happy about WavPack-generated RF64 floats, but not ffmpeg-generated RF64 floats. Hm.