Skip to main content

Topic: Porting Wavpack decoder to Dart (Read 9256 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • darkbyte
  • [*][*][*]
Porting Wavpack decoder to Dart
Hi!

I'm trying to port the Wavpack decoder to Dart. I've used the Java version as the starting point and i've actually made it compilable very quickly. I'm trying to test it first with the standalone Dart VM and when that's working i will make it compatible with in-browser running.

When i started debugging the code i've realized that bit shifting in Dart is not working the same way as in Java because Dart have just one fixed point integer format. It uses a similar numeric system as Javascript. (there's a signed int type for generic integer handling, afaik its a 64bit signed integer, and num for double precision). The decoder fails very quickly in the beginning when it tries to calculate header's CRC. As i see there's a lot of bit manipulations in the code so i guess there's plenty of this kind of errors awaiting for me.

I'm wondering if there's a floating point version of the decoder available? Maybe that would be easier to port. I guess an even better way would be to understand the format and write a Dart decoder from scratch but that's a lot of work although it would be very interesting.

What do you think?

Thanks!

Mod: i'll upload my current code to a code hosting site tonight if i won't forget it.
  • Last Edit: 30 January, 2013, 06:03:29 AM by darkbyte
WavPack -b4x4hc
Opus --cvbr --bitrate 256 --framesize 5

  • nu774
  • [*][*][*][*][*]
  • Developer
Porting Wavpack decoder to Dart
Reply #1
Maybe you had better read ECMA script spec, where semantics of bit-wise operator (such as logical/arithmetic shift) are defined. And when you are working with 32bit integer arithmetic, maybe you could do either of the following:
Code: [Select]
(some expression) | 0 /* convert to sint32 */
(other expression) >>> 0 /* convert to  uint32 */

  • darkbyte
  • [*][*][*]
Porting Wavpack decoder to Dart
Reply #2
Maybe you had better read ECMA script spec, where semantics of bit-wise operator (such as logical/arithmetic shift) are defined. And when you are working with 32bit integer arithmetic, maybe you could do either of the following:
Code: [Select]
(some expression) | 0 /* convert to sint32 */
(other expression) >>> 0 /* convert to  uint32 */


I'm wondering it this works in Dart aswell. I'll give it a shot tonight. Thanks
WavPack -b4x4hc
Opus --cvbr --bitrate 256 --framesize 5

  • soiaf
  • [*][*]
  • Members (Donating)
Porting Wavpack decoder to Dart
Reply #3
Sounds interesting, looking forward to seeing the end result!
If you download the Haxe decoder from the WavPack website, you'll find a file in the zip file called MyWavPack.js - this contains a WavPack decoder Javascript file generated from the various Haxe files - if Dart is using a similar method as Javascript for bit shifting then perhaps looking at this code may help.

  • darkbyte
  • [*][*][*]
Porting Wavpack decoder to Dart
Reply #4
I couldn't make the bit shifting working in Dart as you suggested.
However i've found a more direct way to get signed fix point integers in Dart.
For eg. if you have a buffer with bytes loaded you can load a signed 32 bit integer this way using the "dart:scalarlist" package:

Code: [Select]
  static int getS32Int(List<int> buffer, int from) {
    final resultList = new Int8List(4)
    ..setRange(0, 4, buffer, from);
    return resultList.asByteArray().getInt32(0);
  }


On the sample Wavpack file i'm working with i've already got till the read_entropy_vars phase of WavPackOpenInputFile().
So i'm making progress but i have to debug the Dart and Java code side by side to validate the calculations.
WavPack -b4x4hc
Opus --cvbr --bitrate 256 --framesize 5

  • bryant
  • [*][*][*][*][*]
  • Developer (Donating)
Porting Wavpack decoder to Dart
Reply #5
This is very cool...please keep us posted on your progress (and feel free to post any questions that come up)!

David

  • darkbyte
  • [*][*][*]
Porting Wavpack decoder to Dart
Reply #6
To revive this topic, i've uploaded my latest attempt here before i completely lose it: wv4dart.zip
Sadly i don't had time to go on with it. And i felt that without understanding the inner workings of the codec it's not a trivial task to find out what's going wrong. It's a very simple CLI app for now which can be run using the standalone Dart VM or with DartEditor. No prompt or help yet but it works similarly to the example of the Java version.

I think it's worth to give it another try. Especially that now Dart supports SIMD operations and i'm interested how big speed boost can be achieved with that.

bryant, is there any documentation which details the inner workings of the coded (biased on the decoder)? So far i found your math theory and the file format documentation. Is there any "noobish" documentation which explains the structure of the decoder? DSP is a very new territory for me. 

Thanks!
  • Last Edit: 25 June, 2013, 06:01:08 PM by darkbyte
WavPack -b4x4hc
Opus --cvbr --bitrate 256 --framesize 5

  • saratoga
  • [*][*][*][*][*]
Porting Wavpack decoder to Dart
Reply #7
https://github.com/FFmpeg/FFmpeg/blob/maste...codec/wavpack.c

The ffmpeg decoder is quite easy to understand if you need another reference.

  • darkbyte
  • [*][*][*]
Porting Wavpack decoder to Dart
Reply #8
The ffmpeg decoder is quite easy to understand if you need another reference.


Thanks saratoga!
WavPack -b4x4hc
Opus --cvbr --bitrate 256 --framesize 5

  • soiaf
  • [*][*]
  • Members (Donating)
Porting Wavpack decoder to Dart
Reply #9
Thanks for posting that. I tried out your code (needed to convert scalarlist to typed_data to get it to run) - if I find some time over the weekend I'll have a look at your code and see if I can determine where it's going wrong, would certainly be nice to get it up and running!

  • darkbyte
  • [*][*][*]
Porting Wavpack decoder to Dart
Reply #10
Maybe we should move the code to a repository then to make the joint development even easier.
I'll create a Github repo soon.

Mod: here it is: https://github.com/nistvan86/wv4dart/

I know that the layout of the packaging is not following the Dart conventions at all but it was just a quick hack. We can refactor it later.

The main problem is the conversion of the bit shift operators and the nearly 1:1 move to the different type system of Dart. As far i remember the median calculation is the first noticable part where calculations go wrong. Some variables are growing infinitely on the sample file i was testing with. The decoder did decode the silent part of the file without failing.
  • Last Edit: 26 June, 2013, 12:25:04 PM by darkbyte
WavPack -b4x4hc
Opus --cvbr --bitrate 256 --framesize 5

  • soiaf
  • [*][*]
  • Members (Donating)
Porting Wavpack decoder to Dart
Reply #11

Another good resource for learning about audio codecs is the audio formats reference document available here:

http://sourceforge.net/projects/audiotools...ts%20reference/

  • darkbyte
  • [*][*][*]
Porting Wavpack decoder to Dart
Reply #12
Another good resource for learning about audio codecs is the audio formats reference document available here:

http://sourceforge.net/projects/audiotools...ts%20reference/


Amazing!  Thanks for sharing, this seems very helpful.
WavPack -b4x4hc
Opus --cvbr --bitrate 256 --framesize 5

  • darkbyte
  • [*][*][*]
Porting Wavpack decoder to Dart
Reply #13
soiaf fixed the decoder this weekend so decoding is now possible. Merged his modifications to my repository so feel free to check it out.
It's still very slow (a 7 minute wavpack required eta. 10 minutes to decode) and some distortions are present in the resulting WAV file, but we have a starting point

Thanks soiaf!
WavPack -b4x4hc
Opus --cvbr --bitrate 256 --framesize 5

  • soiaf
  • [*][*]
  • Members (Donating)
Porting Wavpack decoder to Dart
Reply #14
No problem, glad I could help out.
I'll have a look again sometime over the next few days and see if I can fix a couple more bugs!