Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: TAK 2.3.3 Beta thread  (Read 49315 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

TAK 2.3.3 Beta thread

    Beta release 3 of TAK 2.3.3 ((T)om's lossless (A)udio (K)ompressor)

    TAK 2.3.3 brings an 64-bit decoding library and better unicode support for the GUI version.

    New in Beta 3

    The bug that has been fixed in beta 2 hid another bug, which unfortunately only occured on some systems (not on my test system). It would crash the application. It is fixed in the new 64-bit library.

    New in Beta 2

    The interface of the 64-bit decoder library contained a bug. This release brings a new library and a modified interface for the pascal version which is technically backwards compatible. The C interface remains unchanged. The SDK documentation has been updated.

    It consists of:

    • TAK Applications 2.3.3 Beta 1.
    • TAK Winamp plugin 2.3.3 Beta 1
    • TAK Decoder library x86 (Beta 1) / x64 (Beta 3)
    • TAK SDK 2.3.3 Beta 2

    Re: TAK 2.3.3 Beta thread

    Reply #1 –
    What's new

    New features:

    • An 64-bit decoder library for the SDK.
    • I have also created 64-bit versions of the applications. As expected they are bigger and slower without any advantage. As long as Windows supports 32-bit applications i see no reason to release them. But i will continously maintain them, so that they are ready when needed.
    • Unicode support of the GUI version is no longer limited to the open file dialogs. The required switch to a newer version of my development environment is responsible for a 3.5 times bigger program file.

    Improvements:

    • Tiny encoding speed improvements of not more than 3 percent for Intel cpus based upon the skylake microarchitecture (6th to 10th Generation Core). I could have squeezed out more but only at the expense of significantly slower processing on older platforms. As rule of thumb i am taking into account cpu microarchitectures of the last 10 years.

    Beta testing

    For most users this release is not directly relevant. The most important feature is the 64-bit decoder library. Quite useless for the average user unless a developer incorporates it into his software. Relevant if a software dropped 32-bit support. If you know of such a case, please inform the author about the new library and / or report it here. I already contacted the author of the great Mp3Tag.

    The new library passed my Delphi (Pascal) test set. I also ported some of my tests to C (Code::Blocks environment) to test the header file. Since i am not really fluid in writing C code, i am always a bit uncertain if i did everything right.

    Thanks for testing and have fun

    Thomas

    Re: TAK 2.3.3 Beta thread

    Reply #2 –
    Results

    Here the results for my primary file set.

    Test system: Intel i3-8100 (3.6 GHz / 1 Thread), Windows 10.

    Code: [Select]
    Preset  Enco-Speed                Deco-Speed
    ---------------------------------------------------------
            2.3.2    2.3.3    Win %   2.3.2    2.3.3    Win %
    ---------------------------------------------------------
    -p0     868.72   893.94    2.90   806.84   810.57    0.46
    -p0e    698.08   711.16    1.87   812.44   817.09    0.57
    -p0m    418.70   425.54    1.63   815.07   819.19    0.51
    -p1     726.44   748.04    2.97   787.03   788.37    0.17
    -p1e    482.99   491.72    1.81   788.73   791.12    0.30
    -p1m    314.46   322.68    2.61   790.61   794.60    0.50
    -p2     580.58   591.20    1.83   715.05   718.12    0.43
    -p2e    347.07   348.35    0.37   714.52   719.10    0.64
    -p2m    203.02   206.26    1.60   715.99   718.71    0.38
    -p3     301.25   306.83    1.85   697.20   703.97    0.97
    -p3e    241.91   244.79    1.19   697.88   702.06    0.60
    -p3m    131.89   133.65    1.33   699.12   701.40    0.33
    -p4     183.23   186.54    1.81   650.44   657.64    1.11
    -p4e    158.75   160.61    1.17   651.01   658.40    1.14
    -p4m     82.49    83.73    1.50   651.21   656.84    0.86
    ---------------------------------------------------------
    Speed as multiple of realtime playback.

    And to illustrate the speed disadvantage of the 64-bit version:

    Code: [Select]
    Preset  Enco-Speed                Deco-Speed
    ---------------------------------------------------------
            32 bit   64 bit   Win %   32 bit   64 bit   Win %
    ---------------------------------------------------------
    -p0     893.94   827.49   -7.43   810.57   700.80  -13.54
    -p0e    711.16   661.05   -7.05   817.09   709.15  -13.21
    -p0m    425.54   398.79   -6.29   819.19   710.48  -13.27
    -p1     748.04   698.29   -6.65   788.37   689.93  -12.49
    -p1e    491.72   461.79   -6.09   791.12   693.09  -12.39
    -p1m    322.68   303.56   -5.93   794.60   695.48  -12.47
    -p2     591.20   557.27   -5.74   718.12   634.55  -11.64
    -p2e    348.35   336.29   -3.46   719.10   633.90  -11.85
    -p2m    206.26   193.64   -6.12   718.71   636.49  -11.44
    -p3     306.83   287.58   -6.27   703.97   619.50  -12.00
    -p3e    244.79   233.77   -4.50   702.06   620.18  -11.66
    -p3m    133.65   123.27   -7.77   701.40   621.06  -11.45
    -p4     186.54   172.30   -7.63   657.64   577.94  -12.12
    -p4e    160.61   145.73   -9.26   658.40   579.34  -12.01
    -p4m     83.73    76.29   -8.89   656.84   578.45  -11.93
    ---------------------------------------------------------
    Speed as multiple of realtime playback.

    Re: TAK 2.3.3 Beta thread

    Reply #3 –
    Thanks for the new 64-bit DLL. I used it without trouble in a foobar2000 component.

    For some reason I see much slower decoding with it than I expected. Same component with the 32-bit DLL reaches 1111x realtime speed, the 64-bit component is limited to 400x realtime speed. 64-bit foobar2000 with ffmpeg decodes at 650x realtime speed.
    Tested on Intel Core i9-12900K.

    Re: TAK 2.3.3 Beta thread

    Reply #4 –
    Great news that it is working! And a mistery...

    I have no idea how the 64-bit version could be so slow. I never test the speed of the deco_lib, because it is using the same code as the application with a very thin layer on top. I will write a test program to check if i can replicate your results.

    Can you tell me, which encoder preset is affected?

    Re: TAK 2.3.3 Beta thread

    Reply #5 –
    Quote from: TBeck on
    Can you tell me, which encoder preset is affected?
    The results of the extremes -p0 and -p4m would be interesting.

    Re: TAK 2.3.3 Beta thread

    Reply #6 –
    The test file used p4m compression. I created new test files with the latest beta-encoder with p0 and p4m presets and got these speed results.

    32-bit dll:
    p0:  1379x
    p4m: 1134x

    64-bit dll:
    p0:  961x
    p4m: 408x

    64-bit ffmpeg:
    p0:  871x
    p4m: 653x

    Each decode test was ran three times and I recorded the fastest result. Interestingly the new DLL beats ffmpeg with the p0 file.

    Re: TAK 2.3.3 Beta thread

    Reply #7 –
    Thank you. The 32-bit version on your system is 1.43 respectively 2.78 times faster.

    My newly created speed test gives 1.14 and 1.10, as would be to expected.

    Please tell me which options you use when creating the decoder:
        TtakCpuOptions Cpu;
        TtakInt32      Flags;

    I suspect your 64 bit decoder is using no assembler optimizations. If i compare 32-bit with cpuAny with 64-bit with cpuNone i get ratios of 1.40 and 3.62, quite similar to your results.

    Re: TAK 2.3.3 Beta thread

    Reply #8 –
    Cpu option is set to tak_Cpu_Any (1023) and Flags to 0. I set Flags to "tak_ssd_opt_SequentialRead | tak_ssd_opt_CheckMd5" for integrity verification.

    I ran speed tests on the 64-bit DLL with Cpu set to 0:
    p0:  768x
    p4m: 366x

    Re: TAK 2.3.3 Beta thread

    Reply #9 –
    Will check this soon.

    I already wrote that i am not very familiar with (modern) C / C++. I am unsure, if the aligment of fields in a struct could cause problems (especially in 64-bit mode).  From the SDK:
    Quote
    Structures (record, struct) have to be packed; insertion of filling bytes for data alignment is not allowed. All current structures exclusively contain the data types Int32 and Int64. It should be possible to cope with activated alignment by replacing Int64 with two Int32 for low and high. Then all elements have a size of 4.
    But by some googling i stumbled upon a post at stackoverflow:
    Quote
    First of all, structure alignment is not an exact science and may depend on architecture and compiler. In many case, all structure members are padded according the biggest variable (in byte).

    This definitely would mean trouble. Is there a good way to modify the C header file to avoid such problems? Possibly something like the "packed"-attribute in Delphi Pascal?

    Re: TAK 2.3.3 Beta thread

    Reply #10 –
    Quote from: Case on
    Cpu option is set to tak_Cpu_Any (1023) and Flags to 0. I set Flags to "tak_ssd_opt_SequentialRead | tak_ssd_opt_CheckMd5" for integrity verification.
    Can't replicate it. Here anything is wörking as expected. Could you imagine a problem with the header / interface declaration?

    Re: TAK 2.3.3 Beta thread

    Reply #11 –
    I actually had added #pragma pack(1) to the beginning of my tak_deco_lib.h long ago as the documentation stated things need to be byte aligned, IIRC. Removing the pragma command or changing it to for example 4 byte alignment results in the decoder to stop working. Library returns error.

    Re: TAK 2.3.3 Beta thread

    Reply #12 –
    I suppose  "#pragma pack(1) " is compiler specific? Maybe i should put a note at the top of the header file about alignment for those developers not reading the doucumentation as thoroughly as you.

    Regarding the speed problem: Currently i have no idea what is going on. I will go through the code to find a new approach.

    Re: TAK 2.3.3 Beta thread

    Reply #13 –
    BTW: Which function do you use to create the decoder?

    Re: TAK 2.3.3 Beta thread

    Reply #14 –
    I use tak_SSD_Create_FromStream. I sent you PM with a link to full sources.

    Re: TAK 2.3.3 Beta thread

    Reply #15 –
    Got it, thank you. I am just struggling with the (for me) new message system. Sent you two replies, but i am unsure, if you received them.

    Re: TAK 2.3.3 Beta thread

    Reply #16 –
    I saw the first one as I happened to still be online at the time. I hadn't received mail notification about the second one yet but I would have eventually. Replied.

    Re: TAK 2.3.3 Beta thread

    Reply #17 –
    Here are the values of the parameter AOptions which tak_SSD_Create_FromStream () receives from foo_input_tak :

    Play
      Cpu:   0x03A01760  60823392 (dec)
      Flags:  0x00000000  0

    Verify integrity
      Cpu:   0x04BA1600  79304192
      Flags:  0x00000000  0

    Strange. Cpu is garbage.

    Somehow i have the suspicion that we have a nasty case of a program part overwriting data in another place.

    In your class declaration input_tak StreamInfo precedes the Options variable.

        Ttak_str_StreamInfo_V22 StreamInfo;
        TtakSSDOptions Options;

    It could be worthwhile to insert some dummy fields (maybe 16 bytes or so) between the two to check, if some write acceess to StreamInfo is overwriting Options.  And i have to check tak_SSD_GetStreamInfo_V22().

    Re: TAK 2.3.3 Beta thread

    Reply #20 –
    Quote from: Case on
    I saw the first one as I happened to still be online at the time. I hadn't received mail notification about the second one yet but I would have eventually. Replied.
    I'v got it!

    Three functions are affected:

      tak_SSD_Create_FromFile ()
      tak_SSD_Create_FromFileW ()
      tak_SSD_Create_FromStream ()

    The troublesome parameter (a struct) is declared in pascal as:

       const AOptions : TtakSSDOptions;

    And in C:

        const TtakSSDOptions * AOptions,

    Obviously a pointer to the struct should be passed.

    For decades i  made a false assumption about the effect of const when passing a struct in pascal. I assumed it would always be passed by reference (as pointer). Wrong! If it fits into a register, it will be passed by value. Since TtakSSDOptions is 8 bytes large, it fits perfectly in 64 bit mode.

    My only excuse: "It ever worked as i expected!"I never had to question my assumption.

    Sorry for the trouble! I will release a new version tomorrow.

    Re: TAK 2.3.3 Beta thread

    Reply #21 –
    Beta 2 has been added to the first post.

    It hopefully fixes the reported bugs in the 64-bit decoder library.

    Maybe a Mod could delete the Beta 1 attachement? I seem to have no sufficient rights to do it.

    Re: TAK 2.3.3 Beta thread

    Reply #22 –
    Seems i was too hasty... Just received the report it's still crashing. Surprising. It ran so well in my virtual machine. Then i tried it directly on my primary system and it crashed! Good it did, makes it a lot easier to find the bug by trying things out.

    What could be the difference between the systems? First hypothesis: The library gets loaded at a different adress. What if my assembler code is not perfectly relocatable? Switched it off and everything is running fine. Now i have to dig a bit deeper into 64-bit assembly.

    Funny: The bug in beta 1 was likely to disable the assembler code and therfore hid the new bug.

    It's been a long time since i experienced such a bumpy release.

    Re: TAK 2.3.3 Beta thread

    Reply #23 –
    Good news: I fixed a bug in my assembler code that indeed depended on the memory location of the library. Now the library is working well on both of my systems and 'Case' too has reported success. Big thanks for his support!

    I see a pattern: A new day, a new Beta...

    Re: TAK 2.3.3 Beta thread

    Reply #24 –
    Beta 3 has been added to the first post.