Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Porting Wavpack decoder to Dart (Read 13268 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Porting Wavpack decoder to Dart

Hi!

I'm trying to port the Wavpack decoder to Dart. I've used the Java version as the starting point and i've actually made it compilable very quickly. I'm trying to test it first with the standalone Dart VM and when that's working i will make it compatible with in-browser running.

When i started debugging the code i've realized that bit shifting in Dart is not working the same way as in Java because Dart have just one fixed point integer format. It uses a similar numeric system as Javascript. (there's a signed int type for generic integer handling, afaik its a 64bit signed integer, and num for double precision). The decoder fails very quickly in the beginning when it tries to calculate header's CRC. As i see there's a lot of bit manipulations in the code so i guess there's plenty of this kind of errors awaiting for me.

I'm wondering if there's a floating point version of the decoder available? Maybe that would be easier to port. I guess an even better way would be to understand the format and write a Dart decoder from scratch but that's a lot of work although it would be very interesting.

What do you think?

Thanks!

Mod: i'll upload my current code to a code hosting site tonight if i won't forget it.

Porting Wavpack decoder to Dart

Reply #1
Maybe you had better read ECMA script spec, where semantics of bit-wise operator (such as logical/arithmetic shift) are defined. And when you are working with 32bit integer arithmetic, maybe you could do either of the following:
Code: [Select]
(some expression) | 0 /* convert to sint32 */
(other expression) >>> 0 /* convert to  uint32 */

Porting Wavpack decoder to Dart

Reply #2
Maybe you had better read ECMA script spec, where semantics of bit-wise operator (such as logical/arithmetic shift) are defined. And when you are working with 32bit integer arithmetic, maybe you could do either of the following:
Code: [Select]
(some expression) | 0 /* convert to sint32 */
(other expression) >>> 0 /* convert to  uint32 */


I'm wondering it this works in Dart aswell. I'll give it a shot tonight. Thanks

Porting Wavpack decoder to Dart

Reply #3
Sounds interesting, looking forward to seeing the end result!
If you download the Haxe decoder from the WavPack website, you'll find a file in the zip file called MyWavPack.js - this contains a WavPack decoder Javascript file generated from the various Haxe files - if Dart is using a similar method as Javascript for bit shifting then perhaps looking at this code may help.

Porting Wavpack decoder to Dart

Reply #4
I couldn't make the bit shifting working in Dart as you suggested.
However i've found a more direct way to get signed fix point integers in Dart.
For eg. if you have a buffer with bytes loaded you can load a signed 32 bit integer this way using the "dart:scalarlist" package:

Code: [Select]
  static int getS32Int(List<int> buffer, int from) {
    final resultList = new Int8List(4)
    ..setRange(0, 4, buffer, from);
    return resultList.asByteArray().getInt32(0);
  }


On the sample Wavpack file i'm working with i've already got till the read_entropy_vars phase of WavPackOpenInputFile().
So i'm making progress but i have to debug the Dart and Java code side by side to validate the calculations.

Porting Wavpack decoder to Dart

Reply #5
This is very cool...please keep us posted on your progress (and feel free to post any questions that come up)!

David

Porting Wavpack decoder to Dart

Reply #6
To revive this topic, i've uploaded my latest attempt here before i completely lose it: wv4dart.zip
Sadly i don't had time to go on with it. And i felt that without understanding the inner workings of the codec it's not a trivial task to find out what's going wrong. It's a very simple CLI app for now which can be run using the standalone Dart VM or with DartEditor. No prompt or help yet but it works similarly to the example of the Java version.

I think it's worth to give it another try. Especially that now Dart supports SIMD operations and i'm interested how big speed boost can be achieved with that.

bryant, is there any documentation which details the inner workings of the coded (biased on the decoder)? So far i found your math theory and the file format documentation. Is there any "noobish" documentation which explains the structure of the decoder? DSP is a very new territory for me. 

Thanks!

 


Porting Wavpack decoder to Dart

Reply #9
Thanks for posting that. I tried out your code (needed to convert scalarlist to typed_data to get it to run) - if I find some time over the weekend I'll have a look at your code and see if I can determine where it's going wrong, would certainly be nice to get it up and running!

Porting Wavpack decoder to Dart

Reply #10
Maybe we should move the code to a repository then to make the joint development even easier.
I'll create a Github repo soon.

Mod: here it is: https://github.com/nistvan86/wv4dart/

I know that the layout of the packaging is not following the Dart conventions at all but it was just a quick hack. We can refactor it later.

The main problem is the conversion of the bit shift operators and the nearly 1:1 move to the different type system of Dart. As far i remember the median calculation is the first noticable part where calculations go wrong. Some variables are growing infinitely on the sample file i was testing with. The decoder did decode the silent part of the file without failing.



Porting Wavpack decoder to Dart

Reply #13
soiaf fixed the decoder this weekend so decoding is now possible. Merged his modifications to my repository so feel free to check it out.
It's still very slow (a 7 minute wavpack required eta. 10 minutes to decode) and some distortions are present in the resulting WAV file, but we have a starting point

Thanks soiaf!

Porting Wavpack decoder to Dart

Reply #14
No problem, glad I could help out.
I'll have a look again sometime over the next few days and see if I can fix a couple more bugs!