lossyWAV Development

Topic: lossyWAV Development (Read 619326 times) previous topic - next topic

0 Members and 4 Guests are viewing this topic.

lossyWAV Development

Reply #875 – 2008-02-27 20:46:53

Quote from: shadowking on 2008-02-27 16:11:02

... I just don't know where lossywav quality drops sharply (wavpack falls over below 235 k ) ...

From former experiments I guess that's slighty above 300 kbps. In this bitrate range expectations are higher of course than with wavPack lossy's 235 kbps edge. So I think the practical edge - talking only of bitrate - is pretty much where Nick.C has it now with his least demanding quality setting.

lossyWAV Development

Reply #876 – 2008-02-27 22:02:02

Quote from: halb27 on 2008-02-27 20:46:53

Quote from: shadowking on 2008-02-27 16:11:02
... I just don't know where lossywav quality drops sharply (wavpack falls over below 235 k ) ...
From former experiments I guess that's slighty above 300 kbps. In this bitrate range expectations are higher of course than with wavPack lossy's 235 kbps edge. So I think the practical edge - talking only of bitrate - is pretty much where Nick.C has it now with his least demanding quality setting.

I've been listening to some of the tracks from my 10 album test set (processed using v0.7.9, -7) and I have to say that I am happy with the quality of the output. I haven't been especially listening out for problems and have been able to get on with other things while the music is playing in the background.

lossyWAV beta v0.7.9 attached to post #1 in this thread.

[edit] forgot to include quality setting..... [/edit]

lossyWAV Development

Reply #877 – 2008-02-28 01:17:53

I'm sorry if I don't understand the purpose here, but what's the point of creating a "lossy" flac, without any characteristic lossy modeling techniques, at bitrates around 320kbps?

Can anyone (and I mean anyone) possibly ABX the difference in an MP3 at 320kbps? Well, from what I've seen, no, you can't. However, it does seem like lossy flacs at this bitrate are easily ABX-able.

What gives? If lossy flac is inferior to MP3 at the same bitrate, then...? Sorry for my ignorance in this regard.

lossyWAV Development

Reply #878 – 2008-02-28 02:00:15

MP3 is abaxable on certain signals even at 320 k - artificial impulse heavy stuff.. You are right though that there is no point in 'archiving' without some correction files at less than 99.999 % transparent bitrate. There is a point of creating 'medium' settings as they may be more than enough for a certain listener + they can use some correction file mechanism to fully restore the original.

lossyWAV Development

Reply #879 – 2008-02-28 08:09:14

Quote from: The Sheep of DEATH on 2008-02-28 01:17:53

I'm sorry if I don't understand the purpose here, but what's the point of creating a "lossy" flac, without any characteristic lossy modeling techniques, at bitrates around 320kbps?

Can anyone (and I mean anyone) possibly ABX the difference in an MP3 at 320kbps? Well, from what I've seen, no, you can't. However, it does seem like lossy flacs at this bitrate are easily ABX-able.

What gives? If lossy flac is inferior to MP3 at the same bitrate, then...? Sorry for my ignorance in this regard.

The purpose in introducing -4 to -7 is to accept the requests made by Shadowking, Mitch 1 2 and GeSomeone for a lower bitrate preset. Yes, it *may* not be transparent, but it will certainly extend the battery life on your DAP of choice as lower bitrates = less battery drain in reading files and FLAC is already a low power drain decoder. A recent test at anythingbutipod indicates that even at about 380kbps, lossyFLAC will get more battery life than MP3 on one DAP using RockBox. lossyWAV is not competing with MP3 - however it is satisfying to think that the output from -7 is "listenable" to .

-6 and -7 may not pass muster in terms of quality - it remains to be seen from ABX tests by people with good ears. Unfortunately, lossyWAV has only had a small core of ABX testers throughout its development (to those who have taken part, I am extremely grateful!), so settings validation has been a limited exercise.

@halb27 - I tried my 10 album test set at -7 -snr 21 and I got 1.37GB, 350kbps - so -snr 21 is kicking in a lot more than -snr 17. Sounds like a case for some iteration....

lossyWAV Development

Reply #880 – 2008-02-28 13:57:38

Quote from: Nick.C on 2008-02-28 08:09:14

@halb27 - I tried my 10 album test set at -7 -snr 21 and I got 1.37GB, 350kbps - so -snr 21 is kicking in a lot more than -snr 17. Sounds like a case for some iteration....

It's clear that -snr 21 leads to a higher bitrate though I would not have expected to arrive at 350 kbps with -nts 12 -snr 21. Does your current version use the spreading of 22224 for the 64 sample FFT also with these new quality settings? This could be an explanation.
Anyway I'll try your new -7 and -6 setting tonight with high quality regular music (cause that's what we're targeting at with these settings - suboptimal behavior of specific problem samples don't count much here).
Your bitrate targets are fine IMO, and maybe quality is adequate already wih your parameters. Your own listening experience does sound like that.

lossyWAV Development

Reply #881 – 2008-02-28 20:11:33

Quote from: halb27 on 2008-02-28 13:57:38

Quote from: Nick.C on 2008-02-28 08:09:14
@halb27 - I tried my 10 album test set at -7 -snr 21 and I got 1.37GB, 350kbps - so -snr 21 is kicking in a lot more than -snr 17. Sounds like a case for some iteration....
It's clear that -snr 21 leads to a higher bitrate though I would not have expected to arrive at 350 kbps with -nts 12 -snr 21. Does your current version use the spreading of 22224 for the 64 sample FFT also with these new quality settings? This could be an explanation.
Anyway I'll try your new -7 and -6 setting tonight with high quality regular music (cause that's what we're targeting at with these settings - suboptimal behavior of specific problem samples don't count much here).
Your bitrate targets are fine IMO, and maybe quality is adequate already wih your parameters. Your own listening experience does sound like that.

The current version does indeed have 22224 as the 64 sample FFT spreading - all of the presets from 3 to 7 have the same spreading string.

The more I listen to -7 (I'm using -7a which uses the same FFT's as -2, i.e. 64, 256 and 1024 samples), the more I am content to leave this as the least demanding preset.

I hope that your listening tests go well (and thanks for the testing!).

lossyWAV Development

Reply #882 – 2008-02-28 20:44:13

I just finished listening very carefully to 8 full high quality tracks of various genres from my collection which I know pretty well from frequently listening to them.
I used -7, and I did it in foobar ABX mode so I could easily switch listening to the corresponding spot in the original whenever I thought the encoding isn't totally fine. Which happened very, very often - but not because of a real issue but simply because I was very sceptical towards a lossyWAV 320 kbps encoding.

In fact I was totally content with all these encodings. Keep in mind though this wasn't an abx test, but it was a test which assured me that your -7 setting is expected to let me fully enjoy music.
Forget about my remarks concerning -snr and spreading in the HF region. Guess you were directly on the right track. Congratulations.

As shadowking pointed out, these relatively low bitrate settings are especially welcome with your correction file approach.
I personally will stick with -3, but lesson learnt is that I really don't have to worry about -3's quality which I admit I did sometimes (you certainly remember my concern about overlapping).

Wonderful work, Nick.

lossyWAV Development

Reply #883 – 2008-02-28 20:52:45

Quote from: halb27 on 2008-02-28 20:44:13

I just finished listening very carefully to 8 full high quality tracks of various genres from my collection which I know pretty well from frequently listening to them.
I used -7, and I did it in foobar ABX mode so I could easily switch listening to the corresponding spot in the original whenever I thought the encoding isn't totally fine. Which happened very, very often - but not because of a real issue but simply because I was very sceptical towards a lossyWAV 320 kbps encoding.

In fact I was totally content with all these encodings. Keep in mind hough this wasn't an abx test, but it was a test which assured me that your -7 setting is expected to let me fully enjoy music.
Forget about my remarks concerning -snr and spreading in the HF region. Guess you were directly on the right track. Congratulations.

As shadowking pointed out, these relatively low bitrate settings are especially welcome with your correction file approach.
I personally will stick with -3, but lesson learnt is that I really don't have to worry about -3's quality which I admit I did sometimes (you certainly remember my concern about overlapping).

Wonderful work, Nick.

Thank you - I'm really pleased that -7 has provided an acceptable compromise between quality and bitrate.

I think that the lack of problems at this bitrate is largely due to the revised maximum_bits_to_remove process which is re-calculated on a codec_block by codec_block basis - now maximum_bits_to_remove actually takes into account the RMS value of the samples which are going to have their bits removed.

I have been trying hard to come up with a preset which will achieve as low a bitrate as possible while at the same time not introducing glaring artifacts - for my iPAQ-as-a-DAP-with-large-CF-card solution .

I'm in the middle of a large transcode at the moment: 1374 tracks (of 3556), 13.2GB, 317kbps, 99h41m14s duration. Listening to favourites as I go I am continually pleasantly surprised with the outcome.

lossyWAV Development

Reply #884 – 2008-02-28 21:11:36

Quote from: Nick.C on 2008-02-28 20:52:45

... I think that the lack of problems at this bitrate is largely due to the revised maximum_bits_to_remove process ...

Makes me want to reencode my collection (I used 0.7.4 for my last encoding) - no issue with my new hardware finally working.
I remember a recent remark of yours towards caution (I took it as that) for the higher quality presets which I contributed to your current work with -4 and less. Now I can't find it anymore.

Just to make sure: is it safe to use 0.7.9 for a -3 encoding?

ADDED:
OOPs, I found your remark:
[edit] forgot to include quality setting..... [/edit]
What does that mean?

lossyWAV Development

Reply #885 – 2008-02-28 21:25:10

Quote from: halb27 on 2008-02-28 21:11:36

Quote from: Nick.C on 2008-02-28 20:52:45
... I think that the lack of problems at this bitrate is largely due to the revised maximum_bits_to_remove process ...
Makes me want to reencode my collection (I used 0.7.4 for my last encoding) - no issue with my new hardware finally working.
I remember a recent remark of yours towards caution (I took it as that) for the higher quality presets which I contributed to your current work with -4 and less. Now I can't find it anymore.

Just to make sure: is it safe to use 0.7.9 for a -3 encoding?

ADDED:
OOPs, I found your remark:
[edit] forgot to include quality setting..... [/edit]
What does that mean?

When I was talking about being happy with the output, I omitted to include the quality preset that I transcoded at.

I am absolutely happy with v0.7.9 for transcoding, and if no adverse reports come in in the next few days then it will be v0.8.0 RC3!

The -merge parameter works and will recombine the .lossy.wav and .lwcdf.wav files if they are in the same directory (specify the .lossy.wav and -merge in the command line and it will find the .lwcdf.wav file and output a .wav file with added extension stripped off). I imagine that this will be most easily used on whole album .wav files as it would probably be a pain to do it for lots of individual tracks.

lossyWAV Development

Reply #886 – 2008-02-28 21:38:42

Wonderful. A great step towards the final version.
Nonetheless it would be marvellous if we could get some more listener feedback.

I just decided due to these good results to try to abx -4 and maybe lower on my usual problem samples, probably this weekend. Maybe I'll change my mind and I'll use a setting like this for my next encoding.

lossyWAV Development

Reply #887 – 2008-02-28 23:08:47

Hmm, I see. So even at -7, it's not easily ABXable, and it's great on the battery. I also use my Pocket PC as a DAP, so this is welcome news to me to.

One quick question: What are the a, b, c modes for each preset (you mention they correspond to "extra FFT analyses," but what does that mean from a quality/filesize standpoint)? That is, how is 7 different from 7a different from 7c?

[edit]Hey, Pocket PCs support WavPack lossy, don't they? How does this LossyFLAC up to low-complexity WavPack lossy @320kbps?
[edit2] Looks like there's a 1kbps increasing difference in bitrate moving from -7 to -7a to -7c. Also, lossyWAV takes longer as you go down. That's the "extra analysis" for ya. But why the extra bitrate to go with it?

lossyWAV Development

Reply #888 – 2008-02-28 23:17:39

Quote from: The Sheep of DEATH on 2008-02-28 23:08:47

Hmm, I see. So even at -7, it's not easily ABXable, and it's great on the battery. I also use my Pocket PC as a DAP, so this is welcome news to me to.

One quick question: What are the a, b, c modes for each preset (you mention they correspond to "extra FFT analyses," but what does that mean from a quality/filesize standpoint)? That is, how is 7 different from 7a different from 7c?

Quality presets -1, -2 & -3 use 4, 3 and 2 FFT analyses respectively in processing the codec_blocks (-1 = 64, 256, 512 & 1024 samples FFT'; -2 = 64, 256 & 1024 sample FFT's; -3 = 64 & 1024 sample FFT's).

What "a" does is move from the number of FFT analyses used for that quality preset to the adjacent "better" preset, i.e. -3a = same FFT analyses as -2; -3b = same FFT analyses as -1; -3c = 64, 12, 256, 512 & 1024 sample FFT's.

Exception: -3 to -7 use the same FFT analyses, so -7a = same FFT analyses as -2, etc.

Adding FFT analyses is more likely to spot any quiet spots missed by the FFT's already used, and will generally slightly increase the bitrate (see the comparison on page 35 of the thread).

lossyWAV Development

Reply #889 – 2008-02-29 13:41:51

Quote from: The Sheep of DEATH on 2008-02-28 01:17:53

I'm sorry if I don't understand the purpose here, but what's the point of creating a "lossy" flac, without any characteristic lossy modeling techniques, at bitrates around 320kbps?

Perhaps a bit too late, but in this thread you find the start of this experiment. This is kind of "devellopment" thread with a real life implementation of that idea.
Because of the lack of psycho accoustic modeling it might not work reliable at bit rates as low as 320k, however bit rates depend quite a bit on the material to be encoded.

lossyWAV Development

Reply #890 – 2008-02-29 21:18:15

Quote from: GeSomeone on 2008-02-29 13:41:51

Quote from: The Sheep of DEATH on 2008-02-28 01:17:53

I'm sorry if I don't understand the purpose here, but what's the point of creating a "lossy" flac, without any characteristic lossy modeling techniques, at bitrates around 320kbps?

Perhaps a bit too late, but in this thread you find the start of this experiment. This is kind of "devellopment" thread with a real life implementation of that idea.
Because of the lack of psycho accoustic modeling it might not work reliable at bit rates as low as 320k, however bit rates depend quite a bit on the material to be encoded.

Ah, very helpful thread! It should probably even be linked to on the first post, in my opinion.

No psychoacoustic modeling, I see. Does wavpack implement such a model at ~235kbps? If not, perhaps the two are in the same boat after all.

lossyWAV Development

Reply #891 – 2008-02-29 23:50:54

Wavpack now has a basic psychoacoustic mechanism in --dns option as well as smart mid-side stereo through -x. It will shift noise up or down depending on the signal. When using -S0 noise falls flat like lossywav.

lossyWAV Development

Reply #892 – 2008-03-01 04:21:25

Quote from: shadowking on 2008-02-29 23:50:54

Wavpack now has a basic psychoacoustic mechanism in --dns option. It will shift noise up or down depending on the signal. When usign -S0 noise falls flat like lossywav.

I assume implementation of this dynamic noise floor-like algorithm is comparatively difficult (i.e. likely cannot be ported to lossyWAV)? Assuming it can be ported, would it provide as substantial a quality gain in the lower (<320kbps) bitrate ranges? This is quite intriguing.

lossyWAV Development

Reply #893 – 2008-03-01 09:10:38

Is it possible to work with foobar with parameters "correction" and "merge"?

lossyWAV Development

Reply #894 – 2008-03-01 11:38:32

Quote from: amors on 2008-03-01 09:10:38

Is it possible to work with foobar with parameters "correction" and "merge"?

I haven't yet got my head around how to automate the -merge process. The -correction parameter will be able to be used fairly simply with foobar, the -merge parameter will take more work - I'll try to start modifying the batch file tonight.

lossyWAV Development

Reply #895 – 2008-03-01 12:50:11

Quote from: shadowking on 2008-02-29 23:50:54

Wavpack now has a basic psychoacoustic mechanism in --dns option as well as smart mid-side stereo through -x. It will shift noise up or down depending on the signal. When using -S0 noise falls flat like lossywav.

lossyWAV has no specific noise shifting mechanism but a special mechanism which keeps noise especially small in the low to medium frequency range. In a sense the effect is similar.

lossyWAV Development

Reply #896 – 2008-03-02 22:20:57

I finished my abx test.
I used Atemlied, badvilbel, bibilolo, bruhns, dither_noise_test, eig, fiocco, furious, harp40_1, herding_calls, keys_1644ds, Livin_In_The_Future, S37_OTHERS_MartenotWaves_A, triangle-2_1644ds, trumpet, Under The Boardwalk, Blackbird/Yesterday.

I made up my mind to make it easier for a start and did not use -4 but -7.
The result was: in a strict sense I couldn't abx any of these samples.
For bibilolo (sec. 6.7-9.3) however I got at 6/7 and finally 8/10, for bruhns (sec. 4.6-7.

I got 7/8 and finally 8/10. My feeling was that 'Livin' in the Future' (sec. 23.2-25.6) also isn't totally correct but failed miserably to abx it.
But even in these cases where I think someone with better ears can abx them fine the differences to the original are very subtle to me.

I switched to -6 and with this I could not get even suspicious results for bibilolo and bruhns. This time I improved hearing the difference for 'Livin' in the Future':
7/10. The difference is hard to describe: something like the singing isn't done with exactly the same amount of fun as in the original.

Finally using -5 everything was alright to me even with 'Livin' in the Future'.

So even with these demanding samples and a lot of listening effort at least with what I can give -7 provided an excellent result.
Nick, you provided -4 to -7 for the sake of lower bitrate while accepting small deviations from the original. It's a bit too early to say so, but in case no other experience comes up I think there's no need for this differentiation. -7 is it. In fact it's so good that IMO it can become a -3 (or a -4, and we can let -5 or something inbetween -5 and -4 to be the -new -3). With these great result I think we can also lower the -nts demands down to -nts 0 for -2 and also a bit for -1.

Looks like your new RMS orientation has done a great job.

lossyWAV Development

Reply #897 – 2008-03-02 23:26:31

Now that is great!

Time to start work on -8/9/10 then? I guess the best thing to do is keep going lower until the results become easily abx-able. That was your intention with -7 in the first place, right?

Keep up the good work!

Quote from: halb27 on 2008-03-02 22:20:57

I finished my abx test.
I used Atemlied, badvilbel, bibilolo, bruhns, dither_noise_test, eig, fiocco, furious, harp40_1, herding_calls, keys_1644ds, Livin_In_The_Future, S37_OTHERS_MartenotWaves_A, triangle-2_1644ds, rumpet, Under The Boardwalk, Blackbird/Yesterday.

I made up my mind to make it easier for a start and did not use -4 but -7.
The result was: in a strict sense I couldn't abx any of these samples.
For bibilolo (sec. 6.7-9.3) however I got at 6/7 and finally 8/10, for bruhns (sec. 4.6-7. I got 7/8 and finally 8/10. My feeling was that 'Livin' in the Future' (sec. 23.2-25.6) also isn't totally correct but failed miserably to abx it.
But even in these cases where I think someone with better ears can abx them fine the differences to the original are very subtle to me.

I switched to -6 and with this I could not get even suspicious results for bibilolo and bruhns. This time I improved hearing the difference for 'Livin' in the Future':
7/10. The difference is hard to describe: something like the singing isn't done with exactly the same amount of fun as in the original.

Finally using -5 everything was alright to me even with 'Livin' in the Future'.

So even with these demanding samples and a lot of listening effort at least with what I can give -7 provided an excellent result.
Nick, you provided -4 to -7 for the sake of lower bitrate while accepting small deviations from the original. It's a bit too early to say so, but in case no other experience comes up I think there's no need for this differentiation. -7 is it. In fact it's so good that IMO it can become a -3 (or a -4, and we can let -5 or something inbetween -5 and -4 to be the -new -3). With these great result I think we can also lower the -nts demands down to -nts 0 for -2 and also a bit for -1.

Looks like your new RMS orientation has done a great job.

lossyWAV Development

Reply #898 – 2008-03-03 08:34:31

Quote from: The Sheep of DEATH on 2008-03-02 23:26:31

Now that is great!

Time to start work on -8/9/10 then? I guess the best thing to do is keep going lower until the results become easily abx-able. That was your intention with -7 in the first place, right?

Not exactly - but probably a good place to start...

I have revised -1 to -7 as seen in the table below and have processed my 53 sample set:

Code: [Select]

   Preset   [Equiv. Settings]    Total Size      Bitrate  [Delta.BR]     10 Album Test Set
==========================================================================================
    FLAC    [---------------] 72,652,785 bytes, 781.6kbps [--------] 3.35GB, 854kbps (-------)
  v0.7.9 -1 [-nts -4 -snr 25] 52,138,258 bytes, 560.9kbps [--------] 1.94GB, 496kbps (-65kbps)
  v0.7.9 -2 [-nts -2 -snr 23] 48,177,581 bytes, 518.3kbps [42.6kbps] 1.78GB, 453kbps (-65kbps)
  v0.7.9 -3 [-nts  0 -snr 21] 42,976,155 bytes, 462.3kbps [56.0kbps] 1.58GB, 403kbps (-59kbps)
  v0.7.9 -4 [-nts  3 -snr 20] 40,324,698 bytes, 433.8kbps [28.5kbps] 1.47GB, 375kbps (-59kbps)
  v0.7.9 -5 [-nts  6 -snr 19] 37,934,855 bytes, 408.1kbps [25.7kbps] 1.38GB, 352kbps (-56kbps)
  v0.7.9 -6 [-nts  9 -snr 18] 35,826,396 bytes, 385.4kbps [22.7kbps] 1.31GB, 333kbps (-52kbps)
  v0.7.9 -7 [-nts 12 -snr 17] 33,950,736 bytes, 365.2kbps [20.2kbps] 1.25GB, 318kbps (-47kbps)
==========================================================================================
  v0.8.0 -1 [-nts -3 -snr 24] 51,077,948 bytes, 549.5kbps [--------]
  v0.8.0 -2 [-nts  0 -snr 22] 46,198,740 bytes, 497.0kbps [52.5kbps]
  v0.8.0 -3 [-nts  3 -snr 20] 40,331,901 bytes, 433.9kbps [63.9kbps]
  v0.8.0 -4 [-nts  6 -snr 19] 37,943,564 bytes, 408.2kbps [25.7kbps]
  v0.8.0 -5 [-nts  9 -snr 18] 35,840,504 bytes, 385.6kbps [22.6kbps]
  v0.8.0 -6 [-nts 12 -snr 17] 33,969,718 bytes, 365.4kbps [20.2kbps]
  v0.8.0 -7 [-nts 15 -snr 16] 32,360,935 bytes, 348.1kbps [17.3kbps]

I'm just about to listen to new -7 to see just how awful it is.

lossyWAV beta v0.8.0 attached to post #1 in this thread.

lossyWAV Development

Reply #899 – 2008-03-03 09:01:49

Your new v0.8.0 settings are very attractive to me.
A well-spaced differentiation in quality parameters IMO, and everybody's needs should be satisfied by one of these settings.

Notice