lossyWAV 1.1.0 Development Thread.

Topic: lossyWAV 1.1.0 Development Thread. (Read 185322 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

lossyWAV 1.1.0 Development Thread.

Reply #250 – 2008-06-25 02:57:31

I've greatly enjoyed following the progress of this project, with almost continuous updates and all sorts of interesting feedback from all quarters. But I am somewhat ashamed to admit that only this evening have I actually begun playing with lossyWAV in foobar2000 (I had previously only done a few tests here and there, from the command line).

Based on what I've tried this evening, I have a couple of questions. They have probably been answered elsewhere, but I have not yet spotted the answers this evening... and if I've read them in the multitude of lossyWAV pages I've read over the past few months, I've forgotten.

Briefly:

It was very easy to get lossyWAV working using the suggested structure of C:\bin\lossywav, but it is odd to have to place my codecs in a root-level directory. Every other audio codec resides in a "codecs" sub-folder, within my foobar2000 directory. Is the number of accessible directory levels a limitation of calling the process via cmd, as the encoder? Or will this eventually be altered, to allow a complex structure along the lines of "C:\Program Files\assorted audio stuff\foobar2000\codecs\"? (The program seems to work fine if run from a complex directory structure, but only when called directly from the command line.)

Since lossyWAV-encoded Redbook audio is still two channels, with 16 bit wordlength, 1411 kbps, and 44100 Hz, is there any reason lossyWAV could not be used to archive material that will later be burned back to a CD-R? (Effectively, isn't the VBR nature of lossyWAV closer to a "padded" CBR?)

Thank you!

- M.

lossyWAV 1.1.0 Development Thread.

Reply #251 – 2008-06-25 05:29:27

I'm running it from foobar using the following batch file, and it works fine (doesn't need to be at the root):

Code: [Select]

@echo off
D:\library\software\audio\active_encoders_all\lossywav\lossyWAV %1 %3 %4 %5 %6 %7 %8 %9 --below --nowarnings --quiet
D:\library\software\audio\active_encoders_all\lossywav\flac -5 -f -b 512 "%~N1.lossy.wav" -o"%~N2.flac" 
del "%~N1.lossy.wav"

Note: No spaces in my address, whereas:
C:\Program Files\assorted audio stuff\foobar2000\codecs\
does have spaces, and that may be the issue. See here.

C.

lossyWAV 1.1.0 Development Thread.

Reply #252 – 2008-06-25 08:06:06

Quote from: M on 2008-06-25 02:57:31

It was very easy to get lossyWAV working using the suggested structure of C:\bin\lossywav, but it is odd to have to place my codecs in a root-level directory. Every other audio codec resides in a "codecs" sub-folder, within my foobar2000 directory. Is the number of accessible directory levels a limitation of calling the process via cmd, as the encoder? Or will this eventually be altered, to allow a complex structure along the lines of "C:\Program Files\assorted audio stuff\foobar2000\codecs\"? (The program seems to work fine if run from a complex directory structure, but only when called directly from the command line.)

I am becoming more convinced that the "spaces-in-the-path-to-the-executable" problem is in some way due to the way that cmd.exe is being called - I'll spend a bit of time investigating*....

Quote from: M on 2008-06-25 02:57:31

Since lossyWAV-encoded Redbook audio is still two channels, with 16 bit wordlength, 1411 kbps, and 44100 Hz, is there any reason lossyWAV could not be used to archive material that will later be burned back to a CD-R? (Effectively, isn't the VBR nature of lossyWAV closer to a "padded" CBR?)

PCM in a WAV file is always most significant bit (msb) justified. So, a 9 bit sample occupies the top 9 bits in a 16-bit word. All lossyWAV does is vary the number of msb's in use per codec-block while maintaining the original sample size (in bytes). The audio is still fully WAV format compliant and there is no reason not to burn back to CD-R (I assume as CDDA rather than data?).

I'm glad that you're enjoying lossyWAV - I've enjoyed it this far as well - although, without David's publication of his method just over a year ago, it would never have happened.

[edit] * I may have cracked it - try:

Code: [Select]

Encoder: c:\windows\system32\cmd.exe
Extension: lossy.flac
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --low --stdout|c:\"program files"\bin\flac - -b 512 -5 -o%d
Format is: lossless or hybrid
Highest BPS mode supported: 24

[/edit]

lossyWAV 1.1.0 Development Thread.

Reply #253 – 2008-06-25 14:25:10

That seems to have gotten it... thank you! (Also, you are correct in the assumption of CDDA versus data.)

- M.

lossyWAV 1.1.0 Development Thread.

Reply #254 – 2008-06-25 15:47:58

Quote from: Nick.C on 2008-06-25 08:06:06

* I may have cracked it - try:

Code: [Select]

Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --low --stdout|c:\"program files"\bin\flac - -b 512 -5 -o%d

That's another variation, mine was the "short name", just one page ago

Code: [Select]

set FLAC_path="C:\Progra~1\bin\flac"

lossyWAV 1.1.0 Development Thread.

Reply #255 – 2008-06-25 19:30:44

I just tested lossyWAV 1.0.1u RC1 using --standard with my usual problem set as well as some regular tracks.
Everything's fine (as was expected - but there's always the possibility with changes in code that something goes wrong).

lossyWAV 1.1.0 Development Thread.

Reply #256 – 2008-06-26 21:30:17

Quote from: GeSomeone on 2008-06-25 15:47:58

Quote from: Nick.C on 2008-06-25 08:06:06
* I may have cracked it - try:
Code: [Select]
Parameters: /d /c c:\"program files"\bin\lossywav - --standard --silent --low --stdout|c:\"program files"\bin\flac - -b 512 -5 -o%d
That's another variation, mine was the "short name", just one page ago
Code: [Select]
set FLAC_path="C:\Progra~1\bin\flac"

I was thinking about that too; since cmd is a dos-(related) command short filenames could work too.
Or one adds the dir with lossywav, flac, lame and other audio utils to the path:

In Windows XP this can be done in: (iirc)

Control Panel > System > Advanced > Environment Variables > Path > edit..

lossyWAV 1.1.0 Development Thread.

Reply #257 – 2008-06-26 23:39:10

Quote from: GeSomeone on 2008-06-25 15:47:58

That's another variation, mine was the "short name", just one page ago
Code: [Select]
set FLAC_path="C:\Progra~1\bin\flac"

Hmm, if it's because of how cmd is launching, and using 8.3 path in... well... path, fixes it...

Then try setting the encoder to be "C:\WINDOWS\system32\cmd.exe" with those quotes around it. That *should* force the method that Foobar is using to launch cmd to use long filenames. There's still a possibility that Foobar is explicitly forcing 8.3 names, and why that would be is beyond me.

lossyWAV 1.1.0 Development Thread.

Reply #258 – 2008-06-27 17:24:35

Hi all,

Firstly lossyWAV is excellent, really impressed!

Just ran a test on a sample of classical FLACs encoded at -5.
The sample was 110 files that were all between 540 and 560 kbps.
The reason for this selection is that originally I thought that below 550 the savings rarely warrant using lossyWAV (however, this may not be so - but it's tricky to identify which files are suitable and which aren't).

Results:

Files: AVG kbps / Total (MB) (Saving (MB)) / % of orig size
Original FLAC -5: 551 / 2,267 / - / 100%
lossyWAV -7.0 - 535 / 2,203 (64 MB) / 97%
lossyWAV -6.5 - 526 / 2,167 (100 MB) / 96%
lossyWAV -6.0 - 516 / 2,124 (143 MB) / 94%

The saving was less than I expected.
I had a look at a few individual cases and found that a good number of lossyWAV tracks were larger than the originals (from -6 to -7).
So where lossyWAV worked it worked really well, where it couldn't make reductions it increased the file size.

I don't mean this as a criticism of lossyWAV in any way whatsoever, in fact I've been enormously impressed !!!

What is clear is that one cannot tell by the original bitrate whether or not lossyWAV can make a difference. For example I've had good reductions on files < 500 kbps.

So what I'd really like to see is a program that runs the lossyWAV analysis without doing the encoding (like WAVGain).
- User sets a minimum reduction requirement in %.
- Programs identifies all files where that reduction cannot be achieved.
- Then creates an m3u playlist of all the files that can make the user specified reduction.
- The user then loads playlist into encoder UI (i.e. foobar2k) and off you go.

Just a thought / wish.
In the mean time I'm going to do this manually.

By the way, great idea to allow non-integer values for lossyWAV encoding
Very nice.

C.

lossyWAV 1.1.0 Development Thread.

Reply #259 – 2008-06-27 18:11:09

Better to use lossywav -5 as reference . It is 'standard'.

lossyWAV 1.1.0 Development Thread.

Reply #260 – 2008-06-27 19:42:16

Quote from: carpman on 2008-06-27 17:24:35

...Just ran a test on a sample of classical FLACs encoded at -5.
The sample was 110 files that were all between 540 and 560 kbps.
The reason for this selection is that originally I thought that below 550 the savings rarely warrant using lossyWAV ...

Yes, it's a basic problem.
Guess your tracks consists of rather 'simple' music (from a lossless codec's view), that is music which is rather quiet at least at considerable amount of spots and/or is made up from one or few instruments only.

First thought: for this kind of music it's really disputable whether it's worth while not to use a lossless codec (and I guess you'll often find that when replacing FLAC by wavPack -hx3 or -hhx3 or even better TAK -p4 or even better Monkey -c4000 the situation is even more shining for the lossless codec).

Second thought:
The first thing to consider is the lossyWAV quality demand. In case you don't use lossyWAV for archiving there's not much use in going beyond the --standard quality, and in this case you do get some savings with most of your samples as compared to lossless. And probably --standard is already some overkill which you can find out for your needs by listening tests.

If you use lossyWAV as a replacement for lossless archiving like me the situation is worse of course. But if the percentage of tracks where lossyWAV brings a remarkable relief in file size is large (my situation) I don't care about the few tracks which I encode losslessly. And I don't care too much about some tracks where I may have chosen a suboptimal decision.
If on the other hand your collection most of the time consists of such tracks which are easy to encode for a lossless codec I wouldn't care about lossyWAV (though I might save some file size on some tracks) and would use lossless straight ahead.

lossyWAV 1.1.0 Development Thread.

Reply #261 – 2008-06-27 23:48:12

Quote from: shadowking on 2008-06-27 18:11:09

Better to use lossywav -5 as reference . It is 'standard'.

Good point, -5.0 added below:

Files: AVG kbps / Total (MB) (Saving (MB)) / % of orig size
Original FLAC -5: 551 / 2,267 / - / 100%
lossyWAV -7.0 - 535 / 2,203 (64 MB) / 97%
lossyWAV -6.5 - 526 / 2,167 (100 MB) / 96%
lossyWAV -6.0 - 516 / 2,124 (143 MB) / 94%
lossyWAV -5.0 - 491 / 2,022 (245 MB) / 89%

This mini snapshot illustrates the issue very well. In red are the lossyWAV at -5 versions (bitrate at the end):

053. Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 559
054. Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 564
055. Bach J.S. - [Gould #27] Goldberg Variations - Variation No.26 (1955) - 554
056. Bach J.S. - [Gould #27] Goldberg Variations - Variation No.26 (1955) - 359
057. Bach J.S. - [Gould #31] Goldberg Variations - Variation No.30 (1980) - 555
058. Bach J.S. - [Gould #31] Goldberg Variations - Variation No.30 (1980) - 559

On 55 lossyWAV achieves a huge reduction (i.e. 56)
On 53 lossyWAV makes it larger (see 54)
Yet the initial bitrate is very similar.

If there was a tool to identify this prior to encoding, then clearly I'd process 55 but leave 53 alone.

@halb27

Yes, I am using lossyWAV as a replacement for lossless archiving. The reason for the test was to identify the cut-off where it was not useful to employ lossyWAV. There are plenty of files in my classical collection > 550 kbps and lossyWAV has a huge effect and is very worthwhile anywhere between -6 to -7.5 (I've not bothered testing > -7.5).

But, as I said, what was interesting were the many samples that were < 500 kbps where lossyWAV also achieved a significant reduction. So the problem, as I mentioned was not simply a case of saying "ah, below x kbps lossyWAV has no value, thus I shall only use it on files over x kbps". The issue is one of prior identification.

Where a reduction of less than, say, 5% is possible, I'm going to leave them as pure lossless FLACs - that's fine. So far - 6.5 looks good/very safe to me. LossyWAV is still young and I'm sure problem samples will arrive as its take-up increases -- though I may be wrong.

C.

lossyWAV 1.1.0 Development Thread.

Reply #262 – 2008-06-28 02:32:41

lossywav 1.0.1u

flac -x -b 512 --keep-foreign-metadata %1
(-q 10) - 89,96%
(-q 7.5) - 77,15%
(-q 5) - 65,38%
(-q 2.5) - 51,35%
(-q 0) - 34,96%

for example

flac -x %1
(-q 10) - 96,52%
(-q 7.5) - 83,55%
(-q 5) - 70,93%
(-q 2.5) - 56,45%
(-q 0) - 39,69%

lossyWAV 1.1.0 Development Thread.

Reply #263 – 2008-06-28 19:24:16

In case this is in some way related to my last post:
I'm using lossywav 1.0.1u with the wiki settings for foobar (i.e. -b 512)

C.

lossyWAV 1.1.0 Development Thread.

Reply #264 – 2008-06-29 02:43:05

Ran into a strange quirk while using piped input and output:

I first noticed a filesize discrepancy when I was testing stuff from the command line and then from foobar2000.

I used these command lines when testing from the console:

Code: [Select]

wvunpack -q testfile.wv - | lossywav - -q 0 --scale 0.707106781 --silent --stdout | flac - -0 -s -f -b 512 -o test1.lossy.flac

lossywav testfile-fromfb2k.wav -q 0 --scale 0.707106781 --silent --stdout | flac - -0 -s -f -b 512 -o test2.lossy.flac

and this for encoding through foobar2000 and saved as test3.lossy.flac:

Code: [Select]

cmd.exe /d /c lossywav - -q 0 --scale 0.707106781 --silent --stdout | flac - -0 -s -f -b 512 -o %d

I noticed that the output file from foobar was 7.11MB while everything else was 7.02MB, so I stripped tags from all the files using Mp3Tag and saw that nothing changed.

test1.lossy.flac: 7,363,354 bytes (MD5: bd2ceb8a15bb390cd2d2e4e74efca13b)
test2.lossy.flac: 7,363,354 bytes (MD5: bd2ceb8a15bb390cd2d2e4e74efca13b)
test3.lossy.flac: 7,464,204 bytes (MD5: a472581649d91ec91f35ef17ef4806e5)

Can anyone reproduce this or (even better) tell me why this happens? I verified that neither ReplayGain nor DSP was being applied, and that foobar2000 was set to keep lossless sources at original bit depth.

lossyWAV 1.1.0 Development Thread.

Reply #265 – 2008-06-29 20:19:41

Quote from: silverfire on 2008-06-29 02:43:05

Ran into a strange quirk while using piped input and output:
I first noticed a filesize discrepancy when I was testing stuff from the command line and then from foobar2000.

Can anyone reproduce this or (even better) tell me why this happens? I verified that neither ReplayGain nor DSP was being applied, and that foobar2000 was set to keep lossless sources at original bit depth.

And differences in number of seekpoints and/or padding ?

lossyWAV 1.1.0 Development Thread.

Reply #266 – 2008-06-29 21:28:57

Quote from: silverfire on 2008-06-29 02:43:05

Can anyone reproduce this or (even better) tell me why this happens? I verified that neither ReplayGain nor DSP was being applied, and that foobar2000 was set to keep lossless sources at original bit depth.

This was discussed here and it is to do with piped input from foobar2000, basically the seektable is larger than it ought to be.

lossyWAV 1.1.0 Development Thread.

Reply #267 – 2008-06-30 11:06:00

Quote from: carpman on 2008-06-27 23:48:12

This mini snapshot illustrates the issue very well. In red are the lossyWAV at -5 versions (bitrate at the end):

053. Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 559
054. Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 564
055. Bach J.S. - [Gould #27] Goldberg Variations - Variation No.26 (1955) - 554
056. Bach J.S. - [Gould #27] Goldberg Variations - Variation No.26 (1955) - 359
057. Bach J.S. - [Gould #31] Goldberg Variations - Variation No.30 (1980) - 555
058. Bach J.S. - [Gould #31] Goldberg Variations - Variation No.30 (1980) - 559

On 55 lossyWAV achieves a huge reduction (i.e. 56)
On 53 lossyWAV makes it larger (see 54)
Yet the initial bitrate is very similar.

If those numbers in brackets are dates, then I'd guess the 1955 recording is quite hissy, while the 1980 recording is not. Lossless encoders have to preserve this hiss perfectly intact - lossyWAV doesn't.

It could be the default lossyFLAC blocksize isn't appropriate for these recordings. Take the lossyFLAC versions and re-encode them to FLAC, but with default FLAC settings. That might help.

Otherwise, it can simply be that lossyWAV can't find anything to remove.

Quote

But, as I said, what was interesting were the many samples that were < 500 kbps where lossyWAV also achieved a significant reduction. So the problem, as I mentioned was not simply a case of saying "ah, below x kbps lossyWAV has no value, thus I shall only use it on files over x kbps". The issue is one of prior identification.

I think we've had this discussion before - it would be useful to have a tool to simply keep the original if it was smaller (or similar).

Cheers,
David.

lossyWAV 1.1.0 Development Thread.

Reply #268 – 2008-06-30 13:01:59

lossyWAV 1.0.1v RC2 attached to post #1 in this thread.

lossyWAV 1.1.0 Development Thread.

Reply #269 – 2008-06-30 19:03:11

Quote from: 2Bdecided on 2008-06-30 11:06:00

If those numbers in brackets are dates, then I'd guess the 1955 recording is quite hissy

They are, and it is.

Quote from: 2Bdecided on 2008-06-30 11:06:00

It could be the default lossyFLAC blocksize isn't appropriate for these recordings. Take the lossyFLAC versions and re-encode them to FLAC, but with default FLAC settings. That might help.

Tried that - the difference is minimal.

Quote from: 2Bdecided on 2008-06-30 11:06:00

I think we've had this discussion before - it would be useful to have a tool to simply keep the original if it was smaller (or similar).

We have (see below) and it would.

With lossy processes there is normally one frontier, but with lossyWAV there are two:
1) TRANSPARENCY (the traditional problem): what's the lowest we can go and still achieve transparency
2) FILESIZE (a problem unique, I think, to lossyWAV): what kind of original file can defeat lossyWAV, not in terms of transparency but in terms of filesize.

Although the latter is not as important as the former, it is still an issue, as no one expects a lossy process to increase bit rate.

C.

lossyWAV 1.1.0 Development Thread.

Reply #270 – 2008-06-30 19:07:36

Quote from: carpman on 2008-06-30 19:03:11

Although the latter is not as important as the former, it is still an issue, as no one expects a lossy process to increase bit rate.

It's not lossyWAV per se which is increasing the bitrate, rather the -b 512 used in the FLAC encoding command line. For a track which gets bigger, try, as David suggested, re-encoding with FLAC without the -b 512 part of the command line. This will default to the original block length (probably 4096 samples) and the processed file should be the same size or maybe slightly smaller.

lossyWAV 1.1.0 Development Thread.

Reply #271 – 2008-06-30 19:25:20

Yes, apologies Nick.
Increasing bitrate was inaccurate of me.
Results with your amendment to FLAC's default blocksize:

Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 559
Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 564
Bach J.S. - [Gould #22] Goldberg Variations - Variation No.21 (1980) - 554 (default FLAC blocksize)

The point is with such a reduction (559 to 554 with lossyWAV -5) I wouldn't have bothered.
A complimentary tool to pre-scan and identify such tracks, would certainly be welcome. That's really all my point is - and I shall leave it be from now on.

When it comes to practical use of lossyWAV I'm not going to be editing my batchfile each time I get an increase in bitrate ; instead I'm going leave that file alone.

C.

EDIT: It's just hit me that I may have a wrong assumption about lossyWAV:
If, hypothetically, lossyWAV can't do anything to reduce the original, does that mean that its process is lossless?
e.g.
ORIGINAL = 500
lossyWAV -2.5 = 500
lossyWAV -7.5 = 500

1) Will the 2 lossyWAV versions be identical?
2) Will the 2 lossyWAV versions be identical to the original?

lossyWAV 1.1.0 Development Thread.

Reply #272 – 2008-06-30 19:44:57

Quote from: carpman on 2008-06-30 19:25:20

EDIT: It's just hit me that I may have a wrong assumption about lossyWAV:
If, hypothetically, lossyWAV can't do anything to reduce the original, does that mean that its process is lossless?
e.g.
ORIGINAL = 500
lossyWAV -2.5 = 500
lossyWAV -7.5 = 500

1) Will the 2 lossyWAV versions be identical?
2) Will the 2 lossyWAV versions be identical to the original?

Possibly.... - or at least very close to identical.

lossyWAV 1.1.0 Development Thread.

Reply #273 – 2008-06-30 19:59:42

Ah! So lossyWAV is even safer than I thought. This makes sense of halb27's response a while back as to not being too bothered about the lossyWAV processed files that had almost zero reduction. He knew they were practically lossless.

Thanks Nick. That's cleared up a strange misconception on my part.

C.

lossyWAV 1.1.0 Development Thread.

Reply #274 – 2008-07-01 16:58:31

They can be absolutely 100% lossless if lossyWAV judges that this is required.

Try a pure digitally generated sine wave, for example.

Cheers,
David.

Notice