Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Can't get float files to decode properly (Read 7355 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Can't get float files to decode properly

I've been pulling my hair out trying to get decoding of float wavpack files to work correctly.

I'm working off of a custom framework build of the library (4.40) on OS X (I'm not  using the tinydecoder).  I am able to decode all bit depths of integer files successfully (8, 12, 16, 24, and 32), so I don't think it is an issue with my build.

The code I'm using is fairly straightforward; but no matter what I try float files are decoded to nothing but white noise.  I figured I would paste the relevant code excerpt here and see if anyone notices anything that looks funky in the float processing.

I tried swab'ing the buffer, but still got white noise.

It is worth mentioning that the command-line client (wvunpack) produces a silent WAV from the same 32-bit float input file (32bit_float.wv from the test suite).

Code: [Select]
- (void) fillPCMBuffer
{
    void                *writePointer;
    int32_t                inputBuffer [WP_INPUT_BUFFER_LEN];
    uint32_t            sample;
    
    for(;;) {
        UInt32    bytesToWrite                = RING_BUFFER_WRITE_CHUNK_SIZE;
        UInt32    bytesAvailableToWrite        = [[self pcmBuffer] lengthAvailableToWriteReturningPointer:&writePointer];
        UInt32    spaceRequired                = WP_INPUT_BUFFER_LEN /* * [self pcmFormat].mChannelsPerFrame */ * (32 / 8);    
        
        if(bytesAvailableToWrite < bytesToWrite || spaceRequired > bytesAvailableToWrite) {
            break;
        }
                
        // Wavpack uses "complete" samples (one sample across all channels), i.e. a Core Audio frame
        uint32_t samplesRead = WavpackUnpackSamples(_wpc, inputBuffer, WP_INPUT_BUFFER_LEN / [self pcmFormat].mChannelsPerFrame);

        // Handle floating point files
        if(MODE_FLOAT & WavpackGetMode(_wpc)) {
            
            if(127 != WavpackGetFloatNormExp(_wpc)) {
                NSLog(@"Floating point data not scaled to +/- 1.0");
                return;
            }
            
            float    *inputFloatBuffer    = (float *)inputBuffer;
            float    *floatBuffer        = (float *)writePointer;
            float    audioSample            = 0;

            for(sample = 0; sample < samplesRead * [self pcmFormat].mChannelsPerFrame; ++sample) {
                audioSample        = inputFloatBuffer[sample];
                *floatBuffer++    = (audioSample < -1.0 ? -1.0 : (audioSample > 1.0 ? 1.0 : audioSample));
            }

            [[self pcmBuffer] didWriteLength:samplesRead * [self pcmFormat].mChannelsPerFrame * (32 / 8)];
        }
        else {
            float    *floatBuffer    = (float *)writePointer;
            double    scaleFactor        = (1LL << WavpackGetBitsPerSample(_wpc));
            double    audioSample        = 0;

            for(sample = 0; sample < samplesRead * [self pcmFormat].mChannelsPerFrame; ++sample) {

                if(0 <= inputBuffer[sample]) {
                    audioSample = (double)(inputBuffer[sample] / (scaleFactor - 1));
                }
                else {
                    audioSample = (double)(inputBuffer[sample] / scaleFactor);
                }
                
                *floatBuffer++ = (float)(audioSample < -1.0 ? -1.0 : (audioSample > 1.0 ? 1.0 : audioSample));
            }
            
            [[self pcmBuffer] didWriteLength:samplesRead * [self pcmFormat].mChannelsPerFrame * (32 / 8)];
        }

        // EOS?
        if(0 == samplesRead) {
            [self setAtEndOfStream:YES];
            break;
        }        
    }
}

Can't get float files to decode properly

Reply #1 –
Quote from: sbooth on
I've been pulling my hair out trying to get decoding of float wavpack files to work correctly.

Hmm, I don't see anything obvious in your code that could cause that. I assume that this is not an Intel-based MAC, right?

You shouldn't have that check in there for the WavpackGetNormFloatExp() because that might prevent you from working with some files. Just set the OPEN_NORMALIZE flag when you open the file instead. But obviously that's not causing this problem.

What happens when you use the -m option with wvunpack when you unpack the file? If that reports an incorrect value, then there's an obvious problem in my code (that is almost certainly an endian issue). If that's correct, then there must be another issue.

Please stop pulling out your hair! I'm sure we can easily figure out what's going on and I appreciate you helping me with it... 

David

Can't get float files to decode properly

Reply #2 –
Also not releated to your white noise issue, I found a problem in the integer code. Instead of

Code: [Select]
double    scaleFactor        = (1LL << WavpackGetBitsPerSample(_wpc));


I think that

Code: [Select]
double    scaleFactor        = (1LL << (WavpackGetBytesPerSample(_wpc) * 8 - 1));


would be correct.

This should have caused most integer files to be 6 dB soft, and the 12 and 20 bits files should have been loud and badly clipping.

Can't get float files to decode properly

Reply #3 –
Quote from: bryant on
Hmm, I don't see anything obvious in your code that could cause that. I assume that this is not an Intel-based MAC, right?
Correct, this is a PowerBook G4.

Quote
You shouldn't have that check in there for the WavpackGetNormFloatExp() because that might prevent you from working with some files. Just set the OPEN_NORMALIZE flag when you open the file instead. But obviously that's not causing this problem.
OK, thanks.  I thought of it more as a sanity check, since I'm passing 0 as the offset_exp when the file is opened.

Quote
What happens when you use the -m option with wvunpack when you unpack the file? If that reports an incorrect value, then there's an obvious problem in my code (that is almost certainly an endian issue). If that's correct, then there must be another issue.
Here is the output:

Code: [Select]
$ ./wvunpack -m ~/Music/Samples/test_suite/bit_depths/32bit_float.wv 

 WVUNPACK  Hybrid Lossless Audio Decompressor  Darwin Version 4.40.0
 Copyright (c) 1998 - 2006 Conifer Software.  All Rights Reserved.

original md5:  8036f57e0fd91bef33604b2de3fb4430        
unpacked md5:  65e15559bde044c947bebaeb1dc9711b        
MD5 signatures should match, but do not!


So perhaps there is a problem in the library itself?

Quote from: bryant on
Also not releated to your white noise issue, I found a problem in the integer code. Instead of

Code: [Select]
double    scaleFactor        = (1LL << WavpackGetBitsPerSample(_wpc));


I think that

Code: [Select]
double    scaleFactor        = (1LL << (WavpackGetBytesPerSample(_wpc) * 8 - 1));


would be correct.

This should have caused most integer files to be 6 dB soft, and the 12 and 20 bits files should have been loud and badly clipping.


I used this method because I found some sample code on Apple's web site (http://developer.apple.com/documentation/D..._section_5.html) that did a similar thing.  After some googling and reading on cycling74's web site, it seemed to me the consensus is that there is no consensus on the "best" way to perform the conversion.

However, what you say is true; the 12 and 20 bit files did sound a bit like crap

Edit: OK, I missed what your fix did at first.  You are correct- I left my error here for posterity.

Can't get float files to decode properly

Reply #4 –
Quote from: sbooth on
Quote
What happens when you use the -m option with wvunpack when you unpack the file? If that reports an incorrect value, then there's an obvious problem in my code (that is almost certainly an endian issue). If that's correct, then there must be another issue.
Here is the output:

Code: [Select]
$ ./wvunpack -m ~/Music/Samples/test_suite/bit_depths/32bit_float.wv 

 WVUNPACK  Hybrid Lossless Audio Decompressor  Darwin Version 4.40.0
 Copyright (c) 1998 - 2006 Conifer Software.  All Rights Reserved.

original md5:  8036f57e0fd91bef33604b2de3fb4430        
unpacked md5:  65e15559bde044c947bebaeb1dc9711b        
MD5 signatures should match, but do not!


So perhaps there is a problem in the library itself?

Indeed!

In the meantime I did a little googling myself and found that GCC allocates bit-fields the same on big-endian and little-endian machines. Unfortunately, I expected it to mirror the changes in integers and floating-point numbers and am using a bitfield to "define" a 32-bit float. 

When I reversed the bitfield definition in my code, I got the same incorrect MD5 as you get on that file!

If you have the time (and interest) you can also reverse the definition in src/wavpack_local.h and see if this fixes all your problems (both the wvunpack and your player). Just change this:

Code: [Select]
typedef struct {
    unsigned mantissa : 23;
    unsigned exponent : 8;
    unsigned sign : 1;
} f32;


to this:

Code: [Select]
typedef struct {
    unsigned sign : 1;
    unsigned exponent : 8;
    unsigned mantissa : 23;
} f32;


Obviously this will break Intel builds, so I need some generic solution that will probably involve some ugly macros. I see GCC 4 has a pragma for this called reverse_bitfields, but it's a little late. 

Anyway, thanks for finding this problem, and this should let you limp along until I get a real fix.

David

Can't get float files to decode properly

Reply #5 –
Quote from: bryant on
If you have the time (and interest) you can also reverse the definition in src/wavpack_local.h and see if this fixes all your problems (both the wvunpack and your player). Just change this:

Code: [Select]
typedef struct {
    unsigned mantissa : 23;
    unsigned exponent : 8;
    unsigned sign : 1;
} f32;


to this:

Code: [Select]
typedef struct {
    unsigned sign : 1;
    unsigned exponent : 8;
    unsigned mantissa : 23;
} f32;


Obviously this will break Intel builds, so I need some generic solution that will probably involve some ugly macros. I see GCC 4 has a pragma for this called reverse_bitfields, but it's a little late. 

Anyway, thanks for finding this problem, and this should let you limp along until I get a real fix.


Thanks for the quick response!  I made the changes you suggested and was able to decode to WAV successfully with matching md5 sums.

FWIW, I added a conditional guard for __BIG_ENDIAN__ around the reversed bitfield and I believe the code should work for both OS X architectures (assuming it worked on Intel previously, it works for me now on PPC).

Can't get float files to decode properly

Reply #6 –
Quote from: sbooth on
Thanks for the quick response!  I made the changes you suggested and was able to decode to WAV successfully with matching md5 sums.

FWIW, I added a conditional guard for __BIG_ENDIAN__ around the reversed bitfield and I believe the code should work for both OS X architectures (assuming it worked on Intel previously, it works for me now on PPC).

Thanks for trying that and letting me know; I'm glad that it worked!

I think using a conditional based on the endian-ness will work fine for you for now, and I thought about just doing that myself. However, the C spec makes it pretty clear that developers should not depend on the allocation of bitfields to be any particular way, so I think the prudent thing for me to do is switch to macros. Fortunately, it's uglier than it is difficult...