Skip to main content

Topic: Any suggestions on minimising tiny encoder’s heap usage on ARM7? (Read 5423 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • okey
  • [*]
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
I'm working with an LPC2387 chip (72MHz) and a quite limited amount of RAM, and I need to run tinyencoder on it. 'As is', the malloc calls to create buffer space in pack_audio() (in tinypack.c) fail, because there isn't enough heap space. Any suggestions to reduce the heap footprint as much as possible?

The audio data I'm working with is 8KHz 16-bit, and I'm using the TinyEncoder source from the Wavpack download page. I suspect I may be going about this the wrong way since the readme says there is no malloc/free usage...

  • saratoga
  • [*][*][*][*][*]
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
Reply #1
I'm working with an LPC2387 chip (72MHz) and a quite limited amount of RAM, and I need to run tinyencoder on it.


How much RAM?

'As is', the malloc calls to create buffer space in pack_audio() (in tinypack.c) fail, because there isn't enough heap space. Any suggestions to reduce the heap footprint as much as possible?

The audio data I'm working with is 8KHz 16-bit, and I'm using the TinyEncoder source from the Wavpack download page. I suspect I may be going about this the wrong way since the readme says there is no malloc/free usage...


tinypack.c isn't part of the decoder.  Its the example program showing you how to use the library.  It mallocs some large buffers to hold the input data.  If you don't have that much RAM, either don't use the example program (since you probably don't want to actually run it on your device...) or process from smaller buffers.

  • okey
  • [*]
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
Reply #2
How much RAM?

64KiB. Much of which (~36-40KiB) is needed elsewhere.
tinypack.c isn't part of the decoder.  Its the example program showing you how to use the library.  It mallocs some large buffers to hold the input data.  If you don't have that much RAM, either don't use the example program (since you probably don't want to actually run it on your device...) or process from smaller buffers.

Thanks. That's pretty much what I needed to know. I had arrived at the conclusion that some of it was intended as an example, but due to some spectacular mental failure I failed to realise that all of was intended as such. Duh.

  • saratoga
  • [*][*][*][*][*]
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
Reply #3
How much RAM?

64KiB. Much of which (~36-40KiB) is needed elsewhere.


Looking at the rockbox version:

http://www.rockbox.org/wiki/pub/Main/Codec...1_SansaClip.png

You need about 64KB not counting in/out buffers on ARMv4, roughly 40% of which is code space.  If you can put the code segments in a ROM or other memory, you might be able to make this work.

  • okey
  • [*]
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
Reply #4
Looking at the rockbox version:

http://www.rockbox.org/wiki/pub/Main/Codec...1_SansaClip.png

You need about 64KB not counting in/out buffers on ARMv4, roughly 40% of which is code space.  If you can put the code segments in a ROM or other memory, you might be able to make this work.

Cheers. As it happens, I have just done exactly that - shoved the code segments into a ROM and managed to squeeze everything else in. Re-arranged some other bits and pieces so that I can reuse another pair of buffers here. At this stage it appears to work (I'm still testing...).

Is there some (non-trivial) point at which the wv and wvc buffer size (wputils.c) gets too small for proper compression, even when not using hybrid mode? I have observed that it does not compress properly for quite small sizes. Of course, I could just be screwing up again.

  • saratoga
  • [*][*][*][*][*]
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
Reply #5
Is there some (non-trivial) point at which the wv and wvc buffer size (wputils.c) gets too small for proper compression, even when not using hybrid mode? I have observed that it does not compress properly for quite small sizes. Of course, I could just be screwing up again.


I'm not sure how the encoder handles small input sizes.

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
Reply #6
Is there some (non-trivial) point at which the wv and wvc buffer size (wputils.c) gets too small for proper compression, even when not using hybrid mode? I have observed that it does not compress properly for quite small sizes. Of course, I could just be screwing up again.

WavPack is not particularly efficient at encoding small block sizes because of the relatively large headers required, and those two buffer sizes determine the largest blocks that can be created with the tiny encoder. Therefore, you want to make those as large as your system allows, but they can certainly be much smaller than the 64K default. The headers are in the 100-byte or so range, so if you could only afford 10K of buffer, then you would still only be losing about 1% efficiency because of the small blocks.

Obviously if you made them really small (like 1K) then you would start losing significant compression (but it still should work even at that size).

BTW, thanks saratoga! 

edit: typo
  • Last Edit: 10 January, 2012, 01:50:48 AM by bryant

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
Reply #7
One other thing to keep in mind is that the headers are larger for the "higher" compression modes, so those modes are less useful at smaller block sizes.

In fact, for very small blocks, the "fast" mode will [unintuitively] provide the best compression!

  • okey
  • [*]
Any suggestions on minimising tiny encoder’s heap usage on ARM7?
Reply #8
WavPack is not particularly efficient at encoding small block sizes because of the relatively large headers required, and those two buffer sizes determine the largest blocks that can be created with the tiny encoder. Therefore, you want to make those as large as your system allows, but they can certainly be much smaller than the 64K default. The headers are in the 100-byte or so range, so if you could only afford 10K of buffer, then you would still only be losing about 1% efficiency because of the small blocks.

Obviously if you made them really small (like 1K) then you would start losing significant compression (but it still should work even at that size).

Ah. By "too small", I mean I tried making them 256 bytes. The results of which now make perfect sense when considering the header size. Thanks very much for the explanation.