Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Yalac - Comparisons (Read 208323 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Yalac - Comparisons

Reply #150 –
Quote from: Synthetic Soul on
I will run the usual tests at home as always; however, as always, these will take me a few days to complete.

Please wait a bit! Joseph Pohm has just reported some decoding errors i want to fix first.

No need to worry: I am quite sure that i allready know the reason and the fix will not significantly affect the compression performance. Somehow i allready had the feeling, that some slight modification necessary for the PreFilter could give some trouble. But my test corpus and the sample files of Josef decoded well hence i released the V0.08.

Quote from: Synthetic Soul on
In the meantime I have been running a test here at work on the corpus I used for the FLAC testing I performed (28 files, with a better compression spread than my 50 file corpus I use for my Yalac testing).  I can't provide timings, as I have been running the test in low priority.  I need some CPU cycles to at least make it look like I'm working...

Anyway, here are the compression results:

Code: [Select]
Setting    Compressed
=====================
High -c0      57.891%
High          57.638%
High -c2      57.631%

Nice to see some improvements with the PreFilter.

Thanks

  Thomas

Yalac - Comparisons

Reply #151 –
The tests at home have nearly completed.  My decoding batch files decode the YAA files and record the time taken; once this is done they then check the MD5 hash of the decoded WAVs against the MD5 hashes of the source files (using FSUM).

This morning I noticed a problem with file "13.wav".  I bought the file into work and have just encoded and decoded it again with all settings.  These are the results:

Code: [Select]
File        MD5
============================================
13.wav      e3df9b6dc143cb473f66091ad5208681

F.wav       e3df9b6dc143cb473f66091ad5208681
N.wav       e3df9b6dc143cb473f66091ad5208681
H-c0.wav    e3df9b6dc143cb473f66091ad5208681
H.wav       e3df9b6dc143cb473f66091ad5208681
H-c2.wav    e3df9b6dc143cb473f66091ad5208681
E-c0.wav    2040c93a2ea07f2b4d5b4b41b5e770b0
E-c1.wav    2040c93a2ea07f2b4d5b4b41b5e770b0
E.wav       2040c93a2ea07f2b4d5b4b41b5e770b0
I-c0.wav    e3df9b6dc143cb473f66091ad5208681
I-c1.wav    e3df9b6dc143cb473f66091ad5208681
I.wav       e3df9b6dc143cb473f66091ad5208681

As you can see, this demonstrates what I briefly noticed at home this morning: There is an issue when encoding this file with the preset "Extra".

I didn't have the time to do a thorough examination this morning, and I didn't have all the results in, but with the data I did have this was the only file and the only settings that had a problem.  I have run the same decoding batch file on the "FLAC" corpus I have here at work and no files or settings have an issue.  Therefore, out of 78 files encoded using 11 settings this is the only evidence I have.

The Yalac_diag.txt report can be found here.

Let me know if I can provide you with any more information.

Quote from: TBeck on
Please wait a bit!
Too late, by the time I get home today all data should be in.

Quote from: TBeck on
Joseph Pohm has just reported some decoding errors i want to fix first.
Ah, I'm glad it's not just me (see above, posted before I had seen this response), and I'm glad you already have an idea what might be happening.

Quote from: TBeck on
Nice to see some improvements with the PreFilter.
Yes, the Fast pre-filter definately seems worth a go.  Until I can look at my timings then I can't see what performance hit there is, but from looking at Shade[ST]'s results it doesn't appear to be much at all, if any.
I'm on a horse.

Yalac - Comparisons

Reply #152 –
Quote from: Synthetic Soul on
...
This morning I noticed a problem with file "13.wav".  I bought the file into work and have just encoded and decoded it again with all settings.  These are the results:
...
As you can see, this demonstrates what I briefly noticed at home this morning: There is an issue when encoding this file with the preset "Extra".
...
I didn't have the time to do a thorough examination this morning, and I didn't have all the results in, but with the data I did have this was the only file and the only settings that had a problem.  I have run the same decoding batch file on the "FLAC" corpus I have here at work and no files or settings have an issue.  Therefore, out of 78 files encoded using 11 settings this is the only evidence I have.
...
Let me know if I can provide you with any more information.

It seems as if i have fixed the error. I could only check one of Josef's files. I will sent him the corrected version and let him try it. If everything seems ok, i will sent the corrections to the other testers. This error should not have affected compression by more than about 0.01 percent.

The error would be more likely to show up on preset EXTRA, hence i am quite confident, that you are suffering from the same error.

Sorry for the inconvenience

  Thomas

Yalac - Comparisons

Reply #153 –
Quote from: TBeck on
It seems as if i have fixed the error. I could only check one of Josef's files. I will sent him the corrected version and let him try it. If everything seems ok, i will sent the corrections to the other testers. This error should not have affected compression by more than about 0.01 percent.

The error would be more likely to show up on preset EXTRA, hence i am quite confident, that you are suffering from the same error.
Ok, thanks Thomas.  Looks like I'm starting my tests all over.

FYI, I have just been looking at the spreadsheet for my "FLAC" corpus, and here are a few more figures for you:

Code: [Select]
                                                 Improvement
Preset: PreFilter                        Average      Best      Worst
=====================================================================
High:   Fast PreFilter                    0.253%    1.024%    -0.004%
High:   Normal PreFilter                  0.260%    1.024%     0.011%

Extra:  Fast PreFilter                    0.258%    1.038%     0.006%
Extra:  Normal PreFilter                  0.261%    1.035%     0.014%

Insane: Fast PreFilter                    0.258%    1.038%     0.005%
Insane: Normal PreFilter                  0.261%    1.036%     0.015%
I'm on a horse.

Yalac - Comparisons

Reply #154 –
Quote from: Synthetic Soul on
FYI, I have just been looking at the spreadsheet for my "FLAC" corpus, and here are a few more figures for you:

Code: [Select]
                                                 Improvement
Preset: PreFilter                        Average      Best      Worst
=====================================================================
High:   Fast PreFilter                    0.253%    1.024%    -0.004%
High:   Normal PreFilter                  0.260%    1.024%     0.011%

Extra:  Fast PreFilter                    0.258%    1.038%     0.006%
Extra:  Normal PreFilter                  0.261%    1.035%     0.014%

Insane: Fast PreFilter                    0.258%    1.038%     0.005%
Insane: Normal PreFilter                  0.261%    1.036%     0.015%

Very interesting! It lightens me to see, that not only Josef's files (which my encoder sometimes finds very special...) can achive a significantly bigger advantage (your best case) than the average.

The main purpose of search level NORMAL is to avoid using the prefilter, if it would hurt. But it's effect seems not very significant.

BTW: I just sent Josef the fixed V0.08a.

Edit: Just learned, that there is a difference between "lightens" and "enlightens"...

Yalac - Comparisons

Reply #156 –
Plans for V0.09

Before i forget:
Quote from: Synthetic Soul on
http://synthetic-soul.co.uk/comparison/yalac/ now compares 0.07 and 0.08 (?all=1 to view all versions tested so far).

http://synthetic-soul.co.uk/comparison/lossless/ now compares 0.08 with other codecs (?all=1 to view the additional prefilter settings).

Many thanks! It's nice to see some improvement with the PreFilter.

Actually i wanted to pause the work on compression improvements after the introduction of the PreFilter and instead begin the work on the file format (error tolerance etc.) to prepare a public release. But some analysis of the PreFilter results has changed this. In addition to its main purpose the PreFilter seems to generate some nice side effects, which help compression in another way than i would have expected. After some thinking i came to the conclusion, that this (and possibly bigger) improvements could be achieved outside of the PreFilter by some other modifications.

A quick and dirty implementation looks promising: About 0.20 percent improvement for preset FAST, 0.30 percent for NORMAL! The encoding speed penality is small for those presets, but because it grows with higher predictor orders may be significant (about 10 percent) for HIGH. The effect on decoding speed is very small.

Because the new method is in no way optimized yet, i would expect some more improvements soon.

But caution: My quick and dirty implementation may contain errors! There is a very small chance, that i am overestimating the possible compression improvements!

Sorry, i can not resist. I have to evaluate this for V0.09.

Yalac - Comparisons

Reply #157 –
No need to apologize TBeck.  There are lots of great lossless encoders available for free right now, so there's no hurry towards a public release.  You do what you need to do to make your encoder the best it can be so that it becomes the new paradigm in Lossless compression!

I have a feeling that your encoder will set a new standard, so we certainly want you to be very methodical, careful, and thoughtful in terms of how you design it, rather than try to rush a release just for the sake of having it publicly available, then try to hack it later with improvements and have to struggle to maintain backwards compatibility and such.

I'm very impressed with your commitment and the consistency that you have been working on this.  You're an inspiration for all developers, for sure!

Yalac - Comparisons

Reply #158 –
Quote from: Supacon on
No need to apologize TBeck.  There are lots of great lossless encoders available for free right now, so there's no hurry towards a public release.  You do what you need to do to make your encoder the best it can be so that it becomes the new paradigm in Lossless compression!

I have a feeling that your encoder will set a new standard, so we certainly want you to be very methodical, careful, and thoughtful in terms of how you design it, rather than try to rush a release just for the sake of having it publicly available, then try to hack it later with improvements and have to struggle to maintain backwards compatibility and such.

I'm very impressed with your commitment and the consistency that you have been working on this.  You're an inspiration for all developers, for sure!

Many thanks!

I appreciate encouraging posts very much (even if i am not always responding)! They are especially helpful, if i just spent several days of work for possible optimizations, which in the end didn't work...

And you are right: I should not hurry. Better develop a well thought design, instead of having to fix bugs or correct shortcomings of a too early public release.

  Thomas
 

Yalac - Comparisons

Reply #159 –
A little late, but I had was requested to  submit the test results conducted on version 0.08a.
Code: [Select]
Alice In Chains - Sap      220,770,524 bytes      duration 20:51
================================================================
name/params		Ratio	EncTime/CPU%	DecTime/CPU%
---------------------	------	------------	------------
Yalacc 0.07 -p0		57.47% 111.97x / 95%   152.51x / 96%
Yalacc 0.08a -p0	57.57%	83.83x / 71%   149.37x / 95%
MAC 4.01 beta2 -c1000	58.33%	66.78x / 97%	55.52x / 91%
FLAC 1.1.2 --fast	63.97%	80.88x / 64%   117.39x / 76%
WavPack 4.3 -f		60.68%	86.65x / 71%   137.56x / 97%
OFR 4.520 --mode fast	57.33%	24.34x / 86%	34.89x / 99%
---------------------	------	------------	------------
Yalacc 0.07 -p1		57.22%	46.68x / 96%   139.48x / 96%
Yalacc 0.08a -p1	57.22%	43.44x / 90%   138.51x / 97%
MAC 4.01 beta2 -c2000	56.87%	50.74x / 97%	43.87x / 94%
FLAC 1.1.2 (default)	59.70%	56.54x / 78%   137.81x / 93%
WavPack 4.3 (default)	59.03%	77.96x / 80%   112.61x / 95%
OFR 4.520 (default)	56.64%	16.99x / 86%	24.83x / 98%
MP4ALS RM17 (default)	57.83%	30.08x / 96%	55.06x / 63%
LA 0.4 normal		55.68%	 6.23x / 99%	 8.06x / 99%
---------------------	------	------------	------------
Yalacc 0.07 -p2		57.09%	17.94x / 98%   123.94x / 96%
Yalacc 0.08a -p2	56.86%	16.93x / 99%   100.71x / 98%
MAC 4.01 beta2 -c3000	56.60%	43.12x / 98%	40.13x / 97%
MAC 4.01 beta2 -c4000	56.24%	23.83x / 98%	23.63% / 99x
FLAC 1.1.2 --best	59.52%	12.08x / 99%   142.47x / 97%
WavPack 4.3 -h		57.36%	52.16x / 90%	73.86x / 98%
OFR 4.520 --best	56.38%	 4.96x / 97%	 6.61x / 99%
MP4ALS RM17 -7		56.57%	 0.97x / 99%	11.64x / 99%
LA 0.4 high		55.49%	 4.55x / 99%	 5.43x / 99%
---------------------	------	------------	------------
Yalacc 0.07 -p2	*	57.09%	17.94x / 98%   123.94x / 96%  (FC /b = OK)
Yalacc 0.08a -p2 -c0 *	57.09%	18.06x / 98%   127.29x / 99%  (FC /b = OK)
Yalacc 0.08a -p2 *	56.86%	16.93x / 99%   100.71x / 98%  (FC /b = OK)
Yalacc 0.08a -p2 -c2 *	56.85%	14.80x / 99%   100.08x / 98%  (FC /b = OK)

Yalacc 0.07 -p3	*	57.03%	11.80x / 99%   132.77x / 97%  (FC /b = OK)
Yalacc 0.08a -p3 *	56.80%	 7.61x / 99%   105.49x / 98%  (FC /b = OK)
Yalacc 0.08a -p3 -c0 *	57.03%	 8.45x / 99%   134.56x / 98%  (FC /b = OK)
Yalacc 0.08a -p3 -c2 *	56.80%	 7.61x / 99%   106.61x / 99%  (FC /b = OK)

Yalacc 0.07 -p4 *	56.97%	 3.39x / 99%   134.34x / 98%  (FC /b = OK)
Yalacc 0.08a -p4 -c0 * 	56.99%	 4.44x / 99%   135.24x / 98%  (FC /b = OK)
Yalacc 0.08a -p4 *	56.76%	 5.26x / 99%   106.47x / 99%  (FC /b = OK)
Yalacc 0.08a -p4 -c2 *	56.76%	 5.26x / 99%   106.75x / 98%  (FC /b = OK)

* Denotes modes with files compared after encoding and decoding


System specs: A64 3000+, 512MB, Caviar 120GB, Win2K
Using -c0 and -c2 switches show a tradeoff with ratio vs. decoding speed, as discussed earlier.

Now, returning to your previous discussion...
"Something bothering you, Mister Spock?"

Yalac - Comparisons

Reply #160 –
Current progress (V0.09)

There is still some more to do before the next evaluation release. But there will be no important changes for the performance. Therefore i like to present some data:

Code: [Select]
Absolute values for test set rw

Enco-Rate    Fastest Fast    Normal  High    Extra                                             
V0.08a                37.93   15.63    6.01    2.69                                             
V0.09         61.23   38.67   17.96    7.17    4.53                                             

Deco-Rate    Fastest Fast    Normal  High    Extra                                             
V0.08a                69.46   57.83   43.79   46.63                                             
V0.09         76.57   69.84   60.39   44.90   46.73                                             

Compression  Fastest Fast    Normal  High    Extra                                             
V0.08a                57.31   56.71   56.20   56.10                                             
V0.09         58.44   57.30   56.64   56.17   56.08                                             

Test system: Pentium 3 / 800 MHz


FASTEST encodes 60 percent faster than FAST. The compression penality is only 1.14 percent on this file set. Decoding is only 10 percent faster. I would expect a bigger advantage, but possibly my system is too slow.

NORMAL encodes 15 and HIGH 20 percent faster.

Sorry, i couldn't resist. I am quite happy, that i could make FASTEST much faster than i had expected.

Yalac - Comparisons

Reply #161 –
I took the liberty of adding your previous 0/07 results.

Code: [Select]
Absolute values for test set rw

Enco-Rate    Fastest   Fast  Normal    High   Extra
V0.07                 37.75   15.74    6.45    3.95
V0.08a                37.93   15.63    6.01    2.69
V0.09         61.23   38.67   17.96    7.17    4.53

Deco-Rate    Fastest   Fast  Normal    High   Extra
V0.07                 68.61   58.37   51.81   53.67
V0.08a                69.46   57.83   43.79   46.63
V0.09         76.57   69.84   60.39   44.90   46.73

Compression  Fastest   Fast  Normal    High   Extra
V0.07                 57.31   56.71   56.36   56.27
V0.08a                57.31   56.71   56.20   56.10
V0.09         58.44   57.30   56.64   56.17   56.08

Test system: Pentium 3 / 800 MHz

No nasty surprises there.  It seems to me to be an excellent mix of additional compression and speed in the faster presets.  The addition of Fastest is extremely welcome, and its stats are extremely impressive.  From my intitial understanding it seems what little compromises there are are all in the right places.

I'm looking forward to comparing it on my corpus; it seems to make it more real when using your own data/files.  From past experience I expect to achieve very similar results though (you often accurately predict the benefits seen by my results when releasing new versions).

Quote from: Destroid on
Quote from: Synthetic Soul on
I would dearly love to hear the opinions of some other, more knowledgeable, members.
Oops, I just read this part. My bad
Ah, better than nothing I guess...
I'm on a horse.

Yalac - Comparisons

Reply #162 –
Quote from: Synthetic Soul on
I took the liberty of adding your previous 0/07 results.

Thanks.

Quote from: Synthetic Soul on
No nasty surprises there.  It seems to me to be an excellent mix of additional compression and speed in the faster presets.  The addition of Fastest is extremely welcome, and its stats are extremely impressive.  From my intitial understanding it seems what little compromises there are are all in the right places.

I hope you don't mind... I would like to increase the predictor order of FASTEST from 8 to 16, which makes encoding 10 percent slower:
Code: [Select]
Enco-Rate    Fastest Fast
V0.09         61.23   38.67
V0.09 16 P    55.93

Deco-Rate    Fastest Fast
V0.09         76.57   69.84
V0.09 16 P    73.88

Compression  Fastest Fast
V0.09         58.44   57.30
V0.09 16 P    58.12

On other test file sets this increase of the predictor order nearly halves the compression difference to FAST. And my P3-800 still encodes 56 * faster than real time...

Yow will be able to evaluate the speed advantage of 8 predictors via command line switches. If you find it important, i may change the preset again.

Yalac - Comparisons

Reply #163 –
As it was with 8 predictors, YALAC looked like my perfect contender.  I'm looking forward to testing it as soon as it's out, to see if results are uniform with what I've expected!

Peace,
Tristan.

Yalac - Comparisons

Reply #164 –
V0.09 is done

Changes:

Compression algorithms:

- New PreFilter option "Sensitivity": Tries to limit the PreFilter usage to those frames, that benefit from it by a significant amount.
- New channel decorrelation method "Difference". While beeing nearly always inferior to the standard decorrelation method, it has two advantages: It's much faster and it compresses better, if both channels contain exactly the same signal (mono signal copied to both channels, seems to happen sometimes).

Presets:

- New preset TURBO, which encodes up to 60 % faster than FAST.
- NORMAL encodes 10 % faster and compresses slightly better.
- HIGH encodes 30 % faster and compresses slightly worse.
- EXTRA encodes 15 % faster.
- INSANE encodes 10 % faster.

Command line:

- New test cases to evaluate the PreFilter-Sensitivity-option:

-c0 = PreFilter off
-c1 = Low
-c2 = Medium
-c3 = High (same sensitivity as in V0.08)

Caution: The preset numbers have changed because of the insertion of TURBO!
0 is now TURBO, 1 = FAST and so on.

Bug fixes:

- Fixed the bug that Synthetic Soul has reported. The smaller frames of the new preset TURBO could easily reproduce it...

Other:

- Clean up of source code. Could give new errors...
- File format changed!

Release:

I hope to send the new version to the following testers within the next 24 hours:

Destroid
Josef Pohm
Shade[ST]
Skymmer
Synthetic Soul

Only reason for this selection: I haven't heard from the other testers within the last 10 days and i don't want to fill their mail box with new versions they possibly currently don't need.

Any of them can sent me an email anytime, and i will sent the current version.

What should be tested:

- Comparison with V0.09: Speed and compression performance.
- Try preset TURBO with 16 instead of 8 predictors: -p04. If this is not encoding significantly slower, we may change the preset.
- Not too important, but you could evaluate the PreFilter Sensitivity: Try preset HIGH with -c0 (off), c1 (Low) and c2 (Medium). No need to check -c3, because this is the default for HIGH.

Plans for V 0.10:

- Try to speed up the PreFilter.
- Possibly evaluate a new method to improve the compression.


Edit: EXTRA encodes only 15 % faster.
Edit 2: Added Skymmer to the tester list.

Yalac - Comparisons

Reply #165 –
Edit: Never mind... You are comparing the new presets with the 0.8 presets, I was confused.
So what is the new distribution of compression speed?

Yalac - Comparisons

Reply #166 –
Code: [Select]
Red Hot Chili Peppers - Mother's Milk    475,939,004 bytes    duration 44:57
============================================================================
name/params		Ratio	EncTime/CPU%	DecTime/CPU%
---------------------	------	------------	------------
Yalacc 0.08a -p0	64.21%	79.14x / 67%	90.00x / 55%
Yalacc 0.09 -p0		64.89%	98.80x / 57%	93.05x / 56%
Yalacc 0.09 -p04	64.27%	88.38x / 66%	89.54x / 56%
Yalacc 0.09 -p1		64.20%	81.19x / 69%	90.85x / 56%

MAC 4.01 beta2 -c1000	64.54%	63.95x / 95%	51.34x / 86%
FLAC 1.1.2 --fast	69.38%	83.23x / 67%	85.15x / 54%
WavPack 4.3 -f		66.44%	79.62x / 67%	84.78x / 61%
OFR 4.520 --mode fast	63.64%	24.20x / 86%	34.63x / 98%
---------------------	------	-------------	------------
Yalacc 0.08a -p1	64.00%	42.74x / 87%	87.35x / 59%
Yalacc 0.09 -p2		63.95%	49.74x / 84%	85.61x / 56%

MAC 4.01 beta2 -c2000	63.70%	49.25x / 97%	41.58x / 91%
FLAC 1.1.2 (default)	65.89%	55.77x / 81%	83.67x / 59%
WavPack 4.3 (default)	65.42%	75.01x / 78%	76.85x / 66%
OFR 4.520 (default)	63.23%	17.63x / 90%	24.83x / 99%
MP4ALS RM17 (default)	64.93%	29.33x / 94%	54.09x / 63%
LA 0.4 normal		62.72%	 6.19x / 99%	 7.84x / 99%
---------------------	------	-------------	------------
MAC 4.01 beta2 -c3000	63.57%	42.83x / 98%	37.27x / 91%
MAC 4.01 beta2 -c4000	63.36%	23.74x / 99%	23.07x / 97%
FLAC 1.1.2 --best	65.73%	11.87x / 99%	82.04x / 57%
WavPack 4.3 -h		64.23%	50.46x / 87%	64.07x / 87%
OFR 4.520 --best	63.19%	 4.95x / 97%	 6.58x / 99%
MP4ALS RM17 -7		63.92%	 1.21x / 99%	18.20x / 91%
LA 0.4 high		62.64%	 4.57x / 99%	 5.40x / 99%

Yalacc 0.08a -p2 *	63.84%	18.40x / 97%	75.44x / 64%  (FC /b = OK)
Yalacc 0.08a -p3 *	63.73%	 7.95x / 99%	75.81x / 63%  (FC /b = OK)
Yalacc 0.08a -p4 *	63.70%	 5.39x / 99%	76.00x / 63%  (FC /b = OK)
Yalacc 0.09 -p3	-c0 *	63.90%	21.68x / 96%	83.39x / 57%  (FC /b = OK)
Yalacc 0.09 -p3 -c1 *	63.88%	22.86x / 96%	87.04x / 61%  (FC /b = OK)
Yalacc 0.09 -p3	-c2 *	63.82%	22.86x / 96%	79.51x / 60%  (FC /b = OK)
Yalacc 0.09 -p3 *	63.79%	22.79x / 95%	74.14x / 64%  (FC /b = OK)
Yalacc 0.09 -p4	*	63.74%	13.10x / 99%	75.77x / 63%  (FC /b = OK)
Yalacc 0.09 -p5	*	63.70%	 5.79x / 99%	76.00x / 62%  (FC /b = OK)

* Denotes modes with files compared after encoding and decoding

System = A64 3000+, 512MB, Caviar 120GB, Win2K
I hit a disk IO wall encoding with TURBO, but note the lower CPU usage in addition to the speed advantage over FASTEST. Over all, all the presets encode notably faster. I was disappointed that the material I used was harder to reduce than I'd hoped, so even tenths of a percent can be important here.

For the next test I'll have my hard disk and RAM upgrades, we'll see if SATA 1.5Gbps does anything

edit: typo fixed, added other compressors' high scores
"Something bothering you, Mister Spock?"

Yalac - Comparisons

Reply #167 –
Destroid's results are greatly inspiring;  I'll try the same experiment with an Edith Piaf and an Oliver Jones albums;  From files that weren't previously mp3s, this time...

Peace,
Tristan.

Yalac - Comparisons

Reply #168 –
Yalac 0.09 vs The Rest of The World (?all=1 to show non-standard (-pT4; -pH c-0; pH c-1; pH c-2))
http://www.synthetic-soul.co.uk/comparison/lossless/

Yalac 0.09 vs Yalac 0.08 (?all=1 to show all Yalac versions)
http://www.synthetic-soul.co.uk/comparison/yalac/

The IO throttling makes my results for Turbo inconclusive, although I suppose that is an argument to go for 16 predictors instead of 8, and therefore pertinent.

I must say I still like the idea of using 8, otherwise Fastest isn't the fastest it can be.  That said, my IO-limited results are an argument against it, for sure.

I will try to add some CPU-only figures to this post ASAP.

Edit: And lo

Code: [Select]
Setting            Encoding                 Decoding
                   CPU    CPU+IO            CPU    CPU+IO
=========================================================
Turbo          122.71x    81.74x        122.45x    82.57x
Turbo -pT4     112.75x    80.31x        121.24x    82.00x
Fast            83.08x    68.84x        112.10x    79.89x
Normal          39.16x    36.73x         95.71x    75.70x
High            18.12x    17.65x         77.85x    68.08x
High -c2        18.01x    17.55x         85.18x    70.63x
High -c1        18.03x    17.53x         89.59x    70.66x
High -c0        17.86x    17.40x         90.19x    72.09x
Extra           10.75x    10.55x         78.90x    67.97x
Insane           5.05x     4.96x         79.72x    67.09x

Dig these in chart form baby.

NB: All MD5 hashes match, including the troublesome (for 0.8(a)) "13.wav".
I'm on a horse.

Yalac - Comparisons

Reply #169 –
Yalac Turbo, according to my corpus, compresses better than WavPack default and FLAC -8.  It is also faster at both encoding and decoding.

Yalac 0.09 is providing better compression than 0.08 in all presets.

The improvement in High is superb: 0.4% better compression with a decent increase in encoding speed.

Normal compresses slightly better, and is again faster than 0.08.  I would still perhaps like to see it encode faster.

That said, there is currently still a nice spread of encoding speeds (122.71x; 83.08x; 39.16x; 18.12x; 10.75x; 5.05x).

I really can't decide about Turbo.  My heart says 8, but my head says 16.  My stomach on the other hand is saying "Hmm... breakfast"!

Quote from: TBeck on
- HIGH encodes 30 % faster and compresses slightly worse.
Not according to the results so far:

Code: [Select]
Results     0.08(a)     0.09
===============================
Me          63.656      63.617%
Destroid    63.84%      63.79%

You're spot on with the 30% for my CPU+IO results.
I'm on a horse.

Yalac - Comparisons

Reply #170 –
Quote from: Synthetic Soul on
Normal compresses slightly better, and is again faster than 0.08.  I would still perhaps like to see it encode faster.

First, again thanks for your great work! And for providing such welcome results... 

Currently i see no way to speed up NORMAL further without a significant compression penality. We could reduce the maximum predictor order from 128 to 96 or 64. This possibly would not affect your test corpus much, but makes a very noticeable difference on other file sets.

Quote from: Synthetic Soul on
That said, there is currently still a nice spread of encoding speeds (122.71x; 83.08x; 39.16x; 18.12x; 10.75x; 5.05x).

Yes. That's the way i wanted it to be: While you can not predict the compression advantage of higher presets on individual file sets, you should at least be able to easily predict the increase of the compression time.

Quote from: Synthetic Soul on
I really can't decide about Turbo.  My heart says 8, but my head says 16.  My stomach on the other hand is saying "Hmm... breakfast"!

That's a very healthy reaction! And possibly the second best after trowing dices for the deceision...

Quote from: Synthetic Soul on
Quote from: TBeck on
- HIGH encodes 30 % faster and compresses slightly worse.
Not according to the results so far:

Code: [Select]
Results     0.08(a)     0.09
===============================
Me          63.656      63.617%
Destroid    63.84%      63.79%

You're spot on with the 30% for my CPU+IO results.

Well, it was hard do predict. High now uses only 192 instead of 256 predictors. Some files of my test corpus really don't like this. But your test corpus anyway does not benefit from higher predictor orders. And HIGH now tries the simple new difference method for the channel decorrelation, that has been added for TURBO. Possibly it helps sometimes. If so, then V0.10 may compress even a little better, because it too contains the Mid-Side variant of the classic difference decorrelation.

And i performed some very slight modifications on some encoder parts; maybe that they help a bit.

Quote from: Synthetic Soul on
Hmm... the compression rate for High (-c3) (63.617%) seems a little odd, considering -c0 (63.743%), -c1 (63.734%),  and -c2 (63.665%).  I would have expected around 63.6%.  I have checked the batch file and it is definately using -pH.  Any thoughts?  Can there possibly be such a difference between -c2 and -c3 ?

Oh yes!

The channel sensitivity option is a quick and dirty approach to solve this problem: With the default sensitivity (same as with V0.08) the PreFilter is even beeing applied, if it does not help compression at all. Unfortunately this decreases decoding speed.

Now i try to predict the PreFilter advantage and disable it (for an individual frame), if the predicted advantage is below a criterion (percent of improvement) set by the sensitivity level:

High: 0.00 = Use it if it does not hurt...
Medium: 0.3
Low: 1.00

But this prediction is not reliable. The deceision is performed at the PreFilter stage and the PreFilter is in front of the whole processing chain of the encoder. You can not reliable predict it's effect on the successing processing steps!

I have selected the sensitivity parameters by simply trying different values on my test corpus. There is no guarantee, that they work well on other file sets.

But on your and destroids test set they perform as expected:

HIGH (sensitivity): Maximum compression advantage but maximum decoding speed penality too.

MEDIUM: Should provide 50 to 70 percent of the advantage of HIGH, but should half the decoding speed penality compared to OFF.

LOW: Only use the PreFilter on the special files (of Joseph Pohm...), which benefit extremely from its usage. Your files are not among them...

Yalac - Comparisons

Reply #171 –
With regard to Turbo, I have dug out a few CPU-only results for FLAC and WavPack to compare with Yalac.

Code: [Select]
Setting                  Enc        Dec       Comp
==================================================
Yalac Turbo          122.71x    122.45x    65.219%
Yalac Turbo -pT4     112.75x    121.24x    64.926%
Yalac Fast            83.08x    112.10x    64.258%
Yalac Normal          39.16x     95.71x    63.880%
Yalac High            18.12x     77.85x    63.617%
Yalac Extra           10.75x     78.90x    63.566%
Yalac Insane           5.05x     79.72x    63.522%

FLAC -0               94.15x    105.84x    70.734%
FLAC -5               42.24x     96.71x    66.279%
FLAC -8                9.67x     94.84x    66.028%

WavPack -f            68.61x     91.34x    67.095%
WavPack Default       59.23x     78.74x    65.750%
WavPack -h            33.15x     53.12x    64.487%
WavPack -x             3.56x     78.32x    65.144%

Firstly, I was suprised at how fast WavPack -f is; I've never really paid it any attention before.  I assume that it would be comparible with FLAC -2 or -3, at a guess.

But, to the reason for this table.  I was very pleased to see both Turbo settings producing such excellent results, and that Fast isn't far off FLAC -0 (when comparing speed).

I was going to say that the table suggests that 8 predictors aren't required, and that 16 could easily be used.  This does ring true, but you could equally say that another 0.3% compression is not required!  When looking at the faster settings Yalac is providing much better compression than the codecs to which I am comparing it.  The fact that Yalac Turbo is faster, at both encoding and decoding, than FLAC -0, and provides an additional 5% compression, is astounding.  Considering FLAC's upcoming improvement, it is possible that Turbo will no longer compress better than FLAC -8, but we will see; it seems Thomas has even further plans up his sleeve as well.

I guess I would have to say, as long as further processes do not slow down Turbo to any relevant degree, 16 predictors makes most sense.  Why not take that little more compression if it means Yalac is still top of the table?

I have no idea what impact the inclusion of error checking, seeking, and tagging will have.
I'm on a horse.

Yalac - Comparisons

Reply #172 –
Quote from: Synthetic Soul on
...
When looking at the faster settings Yalac is providing much better compression than the codecs to which I am comparing it.  The fact that Yalac Turbo is faster, at both encoding and decoding, than FLAC -0, and provides an additional 5% compression, is astounding.  Considering FLAC's upcoming improvement, it is possible that Turbo will no longer compress better than FLAC -8, but we will see; it seems Thomas has even further plans up his sleeve as well.

I guess I would have to say, as long as further processes do not slow down Turbo to any relevant degree, 16 predictors makes most sense.  Why not take that little more compression if it means Yalac is still top of the table?

Exactly my thinking!

And thanks for the table. And not to forget: for your speed graph from the previous post! It took me some time to find it....

Quote from: Synthetic Soul on
I have no idea what impact the inclusion of error checking, seeking, and tagging will have.

To give an first estimate: About 0.02 percent for seeking and error checking when using 250 ms frames. More for Fast and Turbo because of their smaller frames.

But i don't know, how large the average tagging info is.

Yalac - Comparisons

Reply #173 –
Discussion regarding PreFilter vs Parcor Coefficients moved to Yalac – Evaluation and optimization.  I have tried to make the split as neat as possible.  It's a difficult call, as this thread also doubles at the news update thread; however I have moved 27 posts which are not in any way related to comparing Yalac with other codecs.

Discussion regarding tagging moved to new thread: Yalac: Tagging Scheme, as this will surely require further discussion, although perhaps not quite yet maybe.
I'm on a horse.

Yalac - Comparisons

Reply #174 –
Current Progress (V0.10)

Maybe this will later be moved to "Evaluation and optimization"...

Bad News: The time of significant compression or speed improvements every two weeks is now over!

This had to happen sooner or later...

There is only one thing left, that i will try to possibly improve compression on some problem files. But no guarantee!

For the speed: No matter where i am looking into my code, speed optimizations everywhere. You can expect about 10 percent more speed for the fast presets, when i disable my self check code. And possibly cpu specific optimizations could help a bit, but not enough for me to perform them.

This time the list of improvements is quite small:

Presets

- Removed preset INSANE. With one exception any of it's exclusive options provided to litttle additional compression to pay for the speed penality.
- Combined the power of Partition search level Extra (from INSANE) with the speed of level High, which is used by preset EXTRA. Makes preset Extra less than 10 percent slower, but provides at least 70 percent of the compression advantage of old search level extra. With the PreFilter turned on, Search level High (old Extra) is quite useless, but if someone disables the PreFilter to achieve higher decoding speeds, the new Partition search level High can be very advantegous for some files.
- Preset EXTRA now uses the new (classical) channel decorrelation method Mid-Side. Again; not too useful if the PreFilter is turned on, but very fast and therefore a valuable option for faster presets. Makes Extra about 3 percent slower.

Need for speed...

or not?!

Do we really need a TURBO preset, if even FAST is allready beeing limited by today's hard disk performance? Would it at least be useful for transcoding purposes, if the disk-io allready has been performed by the transcoder software (Well, then it's the transcoder software that has to wait for disk-io, our super fast turbo mode isn't responsible, but does this make a difference?)?

My personal opinion (after some introspection...): That can not convince me. At least not, if we are talking about file encoding, decoding and playback on a PC.

But many of you were talking about possible hardware support and the need to take care for the possible limitations. And this surely is very important!

But for me this are some different things, that also should be handled seperately!

Therefore some new (not really) approach for V0.10:

1) I will keep the the presets FAST, NORMAL, HIGH and EXTRA for archival and playback on a PC.

2) The hardware (player) compatibility should be achieved by the definition of profiles, which limit the maximum cpu and memory requirements for decoding.

Basically the profiles are a additional set of presets, hence some kind of TURBO will still be avail...

V0.10 will contain a preliminary definition of 4 profiles (possibly to many), which may get redifined, when i know more about the limitations of hardware players.

Possibly nothing new or surprising for you, but for me a good way to get out of the never ending speed discussion (within my head)...