Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: lossyWAV 1.2.0 Development Thread (Read 311378 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

lossyWAV 1.2.0 Development Thread

Reply #225
It is a bit simplistic because you need to look at the other parameters that change with quality-level - they are what make up the bit-rate changes along each line.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.2.0 Development Thread

Reply #226
I just finished 3 10 trial ABX tests to see whether I can find something wrong with the lossyWAV 1.1.0 --insane --shaping 0 result of Samba e Amor.
I could not find any hint that something maybe wrong and my ABX results were total misses.
I know it's of very limited value cause my ABX results of losyyWAV 1.1.2e were bad as well though my first ABX test then started off fairly well.
Maybe I'm too much inclined to the clearly more audible noise in the error file of 1.1.2e, but I can't withstand the idea that things may have gone wrong after 1.1.0.

BTW the lossyWAV1.1.0 and 1.1.0b results for Samba e Amor are identical when using --insane --shaping 0 (probably they are always identical). So 1.1.0b can be tried instead of 1.1.0. 1.1.0b is still available in the lossyWAV thread on 1.1.0 (including source).
lame3995o -Q1.7 --lowpass 17

lossyWAV 1.2.0 Development Thread

Reply #227
The *big* difference after v1.1.0b is the fundamental change in the spreading function. I think that I need to re-visit the spreading function in light of mixed reports regarding the apparent quality of v1.1.2e vs v1.1.0b.

I am, as always, grateful for the time and effort expended by those users who find new difficult samples and choose to ABX them.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.2.0 Development Thread

Reply #228
Some general comments...

I'm always delighted that people are willing to spend their time listening and testing in order to provide ABX results.

The correction file tells you nothing about what the noise will sound like on top of the music, but it does show you exactly what lossyWAV is doing. That's its use. Without knowledge of the original music, it's meaningless.


I think it would be worth looking further at -I 1.1.2e, -P 1.1.2e, and -I 1.1.0b (all with --shaping 0).

As you know only too well halb27, it makes little sense (on the face of it) to think there's a problem with -I, but completely fail to hear a problem with -P. In investigating this, it makes more sense to listen to the correction file first, rather than the bit-reduced output - if lossyWAV is working as expected then how can -I be audible but -P be inaudible? (I don't say it's impossible - I'm saying, if that's what's happening, let's find out why). If lossyWAV is not working as expected (e.g. there's an important part where -P adds less noise than -I) then it's hopefully an easy fix (but I'm not hopeful that it's this simple).

As for 1.1.0b vs 1.1.2e (both -I --shaping 0) - a moment by moment comparison of added noise would be interesting.


If you haven't done already, I'd encourage you to read the end of the "24-bit vs 16-bit - any samples that work" thread. lossyWAV 1.1.2 -I --shaping 0 wants to keep 19-bits of the 24-bit recording (the same recording which someone can ABX when converted to 16-bits vs the original 24-bits). So it's not like it's being universally careless!

Cheers,
David.

lossyWAV 1.2.0 Development Thread

Reply #229
@BORK:

I'd welcome very much if you could try version 1.1.0 with your last sample.


Sorry for late reply , will dl it & try soon , I have been following in between work,
will try it, thanks.

lossyWAV 1.2.0 Development Thread

Reply #230
@B0rk: I would be interested if you notice any difference by applying a higher upper frequency limit (--limit <n>, where 16000<=n<=20000), say --limit 17500?


missed this post as well , sorry about that,
3 questions :

-what does that switch do ?
-please give me a Full (including all other needed switches) commandline example for it.
-what version to use ?

Thanks.

lossyWAV 1.2.0 Development Thread

Reply #231
For what it's worth I've tried ABXing Samba e Amor and a solo guitar piece from my own library with and without shaping and with 1.1.0 and with 1.1.2e. I can't hear any difference between any of them. Having said that ABXing is not my forte

lossyWAV 1.2.0 Development Thread

Reply #232
3 questions :

-what does that switch do ?
-please give me a Full (including all other needed switches) commandline example for it.
-what version to use ?
1) It raises the upper frequency limit taken into account when determining the average and lowest bin values during the FFT analyses carried out on the audio data;
2) "lossywav <filename> -S --limit 17500 --shaping 0" would process using --standard and with no noise shaping, taking into account frequencies up to 17.5kHz;
3) 1.1.2e in the first instance, 1.1.0b as well if you can spare the time.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

 

lossyWAV 1.2.0 Development Thread

Reply #233
lossyWAV beta 1.1.2f attached to post #1 in this thread.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.2.0 Development Thread

Reply #234
lossyWAV beta 1.1.2f attached to post #1 in this thread.


Nick Thanks !
Ill do it with this version instead.

Quote from: botface link=msg=0 date=
For what it's worth I've tried ABXing Samba e Amor and a solo guitar piece from my own library with and without shaping and with 1.1.0 and with 1.1.2e. I can't hear any difference between any of them. Having said that ABXing is not my forte


botface thanks.
You & other testers please note - 'failing' this ABX Test means nothing.
it dependes on what you focus on , how fast you switch between samples,
& cannot be done without some practice, even if the differences do exist.

Look at this Log I just tested,just got a new dac here to test  - had I stopped in a 11 run, it would have probably been the best result I ever got.
Since I insist to stick with at least a 20 run , & fatigue kicked on me from there on,
I 'failed' it.

but note the 1st 11 first/fresh results.
00:52:33 : 09/11  3.3%

so even though the score is 'failed' that's all I need to know 2 things:
1 - There's a difference , that can be pointed at.
2 - This DAC's Nice

anyone can fail these kind of tests - many times over - it still does not mean that thre is no difference or that long term listening won't hint at it.

Also try different kinds of music - & try to listen to shortish lengths for this test.

foo_abx 1.3.3 report
foobar2000 v0.9.6.2 beta 2
2009/02/10 00:48:51

File A: F:\MP3\Favourite\TEST FILES\112e Insane Shaping 0\Country Roads - Pat Metheny.flac
File B: F:\MP3\Favourite\TEST FILES\112e Insane Shaping 0\Country Roads - Pat Metheny.lossy.flac

00:48:51 : Test started.
00:49:01 : 00/01  100.0%
00:49:25 : 01/02  75.0%
00:49:48 : 02/03  50.0%
00:50:16 : 03/04  31.3%
00:50:31 : 04/05  18.8%
00:50:53 : 05/06  10.9%
00:51:10 : 06/07  6.3%
00:51:25 : 06/08  14.5%
00:51:47 : 07/09  9.0%
00:52:25 : 08/10  5.5%
00:52:33 : 09/11  3.3%
00:52:46 : 09/12  7.3%
00:53:00 : 09/13  13.3%
00:53:25 : 09/14  21.2%
00:53:56 : 09/15  30.4%
00:54:08 : 09/16  40.2%
00:54:25 : 09/17  50.0%
00:54:32 : 10/18  40.7%
00:54:53 : 10/19  50.0%
00:54:59 : 11/20  41.2%
00:55:06 : 11/21  50.0%
00:55:12 : 12/22  41.6%
00:55:23 : 12/23  50.0%
00:55:35 : 13/24  41.9%
00:55:50 : 13/25  50.0%
00:56:17 : 13/26  57.7%
00:56:21 : Test finished.

----------
Total: 13/26 (57.7%)

From Riches To Rags 

lossyWAV 1.2.0 Development Thread

Reply #235
Nick C. , again thanks a lot for the snappy updates,
I really cannot keep up with you ! ( That is NOT a complaint ! You Rock !)

OK I am done testing the old 112e version shaping 0, I just tested one too many of these.

I got some weird results with this DAC I brought in to test with (just to see if it will make any difference for me at all), but some wide spread results,
from results like the above , to 80% wrong on some new file I am not familiar with,
back & forth.

So I just I went back to the OLEO track as I really know it quite well,
wondering If trying an unknown (although better) DAC was a good idea ..
this is probably the only/last time Ill ever see this again, so Im posting it.. :

Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.6.2 beta 2
2009/02/10 01:51:34

File A: F:\MP3\Favourite\TEST FILES\112e Insane Shaping 0\Bob Berg - Oleo.flac
File B: F:\MP3\Favourite\TEST FILES\112e Insane Shaping 0\Bob Berg - Oleo.lossy.flac

01:51:34 : Test started.
01:51:41 : 01/01  50.0%
01:51:47 : 02/02  25.0%
01:51:57 : 02/03  50.0%
01:52:04 : 03/04  31.3%
01:52:16 : 04/05  18.8%
01:52:28 : 05/06  10.9%
01:52:35 : 05/07  22.7%
01:52:56 : 06/08  14.5%
01:53:09 : 07/09  9.0%
01:53:31 : 08/10  5.5%
01:53:48 : 08/11  11.3%
01:54:13 : 08/12  19.4%
01:54:43 : 08/13  29.1%
01:55:01 : 09/14  21.2%
01:55:13 : 10/15  15.1%
01:55:23 : 11/16  10.5%
01:55:39 : 12/17  7.2%
01:55:49 : 13/18  4.8%
01:56:07 : 14/19  3.2%
01:56:28 : 14/20  5.8%
01:56:41 : 15/21  3.9%
01:56:50 : 16/22  2.6%
01:57:00 : 16/23  4.7%
01:57:14 : 17/24  3.2%
01:57:31 : 18/25  2.2%
01:57:44 : 18/26  3.8%
01:58:35 : 19/27  2.6%
01:59:11 : 20/28  1.8%
01:59:33 : 21/29  1.2%
01:59:55 : 22/30  0.8%
02:00:01 : Test finished.

----------
Total: 22/30 (0.8%)


OK, Time for me to move on to the new version & your new commandline instructions,
will continue tomorrw hopefully, Thanks for all the Work You Do !! it's appreciated !

lossyWAV 1.2.0 Development Thread

Reply #236
Nick,

I don't suppose there's a hidden option in lossyWAV to dump the bits_to_remove list into a text file is there?

I'm not asking you to add it in a new version since I'd need it in older versions too to compare, and that would be a mad amount of work for little benefit. I'm just wondering if it's been there all along?

If not, I'll just keep looking at the correction files in Cool Edit. That shows differences easily enough.

Cheers,
David.

lossyWAV 1.2.0 Development Thread

Reply #237
These results are a little hard to understand.

An ABX of -S --shaping 0 was reported here:
http://www.hydrogenaudio.org/forums/index....st&p=613046
...it wasn't that conclusive.

This is the noise added by lossyWAV -S:
[attachment=4882:lossyWAV_S.jpg]


An ABX of -I --shaping 0 has been reported several times.
Some of these ABX results are conclusive.

This is the noise added by lossyWAV -I:
[attachment=4883:lossyWAV_I.jpg]


Given that lossyWAV does absolutely nothing "clever" with the noise when you add "--shaping 0", logic would suggest that -S would be much easier to ABX than -I, since the noise is 13.5dB louder (that's 4x as much noise!).

So what's happening here?

I think it would help greatly if BORG could say more about what he hears in these two situations. So please do!


On the general problem of explaining how 13.5dB more noise may be harder to detect...

It's either a poorly understood properly of human hearing, a poorly understood property of the replay equipment, or some less than rigorous application of ABX statistics.

To have a hope of tracing this problem, we need to attack each possibility in turn, in reverse order I think.


ABX stats

So, with the ABXing, as has already been said, pick a number of trials and stick to it. I doesn't matter what the number is (as long as it's greater than 7), but if you're picking then testing, I'd expect all tests to include the same number of trials.

Also, to save conjecture, take a break when fatigued - don't say "I'd nailed it at 6 trials, but then it went wrong", or "around the 16th trial you'll see I'd got it, but it had gone by the 32nd" etc etc - this is of no more use than tossing a coin, getting 3 heads in a row, and declaring it's a biased coin. If you don't use the statistic properly, they're worthless. It may reflect the reality of testing - people do get fatigued - but somehow we've got to find a way of not breaking the statistics or the tests become worthless.

The idea of doing lots of separate runs of tests because at some point you'll nail it is also worthless. If you head for p=5%, you still have a 5% chance of getting a result by chance. If you repeat the test until you get a positive result, then the (statistical) chance of that positive result being purely down to chance becomes really really high.

It is frustrating and time consuming to have to work like this, and it's a pain because human perception gets tried so easily with repetition - but anything else is just tweaking in the dark.


equipment faults

As in the thread about 16-bit vs 24-bit, the only fault I can think of is that the DAC behaves strangely / specially when LSBs are zeroed. One answer is to fill the bits that would be zeroed with completely random values - i.e. random noise. This is not much use for lossyFLAC encoding, but it is useful for tracking down the source of the audible difference. It's not a completely fair test as the character of the random numbers may not perfectly match the character or power of the truncation noise - but it's a lot closer than the difference between -S and -I, so worth a try.


hearing

Errr, it's about ten years to late to get this into my PhD. Someone else will have to look at this! Maybe it doesn't have to be understood, so much as predicted "well enough" by lossyWAV to avoid audible problems. But it makes sense to rule out the other two factors first I think.

Cheers,
David.

lossyWAV 1.2.0 Development Thread

Reply #238
With the dualstream and wvlossy, The added noise gets louder with lowered bitrate. Its really easy to learn as hiss or noise is the only artifact. Why should Lossywav be different ? The difference should get louder as the quality setting is lowered. That -I is abxed and not -S ?? not even -P ?

The effect is background hiss or (noise - instruments sound 'dusty').. There is nothing else, no thin mids etc etc, no traditional mp3,aac,mpc artifacts. It can be easily confirmed by lowering the quality level. This Oleo track should be tested @ quality 1..2

lossyWAV 1.2.0 Development Thread

Reply #239
Well, yes, so the explanation is...?

Cheers,
David.

lossyWAV 1.2.0 Development Thread

Reply #240
Well I dunno. Even the track oleo is too 'busy' to have real problems for these codecs. LossyW and co like busy action like many instruments = more masking.. right ?

I am not saying the abx is wrong. I think the others have to confirm the problem and that can be difficult. The OP description doesn't match the hiss / noise thing - what is he hearing ? He needs to find out by setting less trials like 8 or 16 at lower quality setting and get a perfect score without fatigue. Then it will be very clear what the difference is. If there are 30 trials + fatigue @ quality <= 3 then I think something is sus.

lossyWAV 1.2.0 Development Thread

Reply #241
Nick,

I don't suppose there's a hidden option in lossyWAV to dump the bits_to_remove list into a text file is there?

I'm not asking you to add it in a new version since I'd need it in older versions too to compare, and that would be a mad amount of work for little benefit. I'm just wondering if it's been there all along?

If not, I'll just keep looking at the correction files in Cool Edit. That shows differences easily enough.

Cheers,
David.
Yes, there is - but it's not hidden: -d or --detail will display a per channel per block bits-to-remove table. Combined with -w or --writetolog and you get a "permanent" record in the form of a lossywav.log file.
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.2.0 Development Thread

Reply #242
equipment faults

As in the thread about 16-bit vs 24-bit, the only fault I can think of is that the DAC behaves strangely / specially when LSBs are zeroed. One answer is to fill the bits that would be zeroed with completely random values - i.e. random noise. This is not much use for lossyFLAC encoding, but it is useful for tracking down the source of the audible difference. It's not a completely fair test as the character of the random numbers may not perfectly match the character or power of the truncation noise - but it's a lot closer than the difference between -S and -I, so worth a try.
I'll use the random number generator, previously used for dither, to provide the necessary random bits - this will be enabled by -r or --randombits.

[edit] lossyWAV beta 1.1.2g attached to post #1 in this thread. [/edit]

[edit2] rounding noise, surely, not truncation noise? [/edit2]
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.2.0 Development Thread

Reply #243
2b Decided - Thaks for the effort & detailed reply.
I have no conclusions or answers to some of the questions raised,
I can provide more info on what I noticed as requested.
Hopefully this thread will generate enough interest to bring in many pros & people with good ears to help LossyWav evolve even further.



Re: Results
-----------
I brought in this DAC so I can test if gear will bring anything different for the testing results here.

This last result with the Oleo Track was done with this borrowed DAC,
with this DAC it was just easier for me to hear the difference in the Midbass thickness, as opposed to the one used before.

(This does not tell me a WHOLE lot as I again tested after a working day,
but that's the time I have now, so that's how it is,
maybe focusing on the lower range was just easier for me because of it ,or maybe this DAC's plain better, not sure.)

I tried to lock on to the noise aspect, but could not really do it,
could be the busy music,being tired.this DAC , no idea.

Thing is ,The real file just sounded fuller in the Mid/Bass region.

What I did not report is that for most of the run, I didnt even bother using the AB buttons.
I thought I locked in on a lack of oomph in the midbass , played X & Y,
decided what's the real file & clicked the answer ( the log times can hint at it).

Re: Runs/Length
--------
The 20 minimum to 40 maximum run choice I use was discussed earlier in this thread,if you recall , when it was suggested that the longer sessions be dropped.
I did drop them, mostly I saw the 20 run usually just tells it like it is.

(Further runs will either weaken or strengthen your score, I get that,
but maybe do have some merit if consecutive positive or negative results are obtained as fatigue does have an effect here , or can prove/disprove a user being able to lock on to it , as sort of additional indication ? you tell me.

I will adjust to anything offered as needed, you decide.


Re: LSBs
--------
The theory of DAC's behaving strangely when LSBs are zeroed,
is interesting , sounds like something that needs to be checked !
but must be first verified in a more scientific way, if you can tell me how to check that myself ?

I mean this is something (at least my) ABX cannot probably be be trusted with,
when all of us I guess, tilted towards one or the other bias polarity.. do not trust these machine-gun type based ABX tests like this 100%).

Is there some way I can test for it ? please advise.


Hearing
--------
The Noise was reported in previous tests if I am not mistaken,
but obviously I only have my current ears & experience to test with.

the first thing Ill do when I can, is test the noise with a non professional user,
that has his hearing intact - with verified good sensitivity up to 20khz,
& see if that at all makes any difference on his subjective uninformed listening impressions, and detailed tests after pointing out the noise differences to him,
& test if he/she can lock onto it.

will report when I can, hope this helps.

lossyWAV 1.2.0 Development Thread

Reply #244
BORK,

Regarding run length, just pick a single number and stick to it. Even saying "it will be between 20 and 40" breaks the statistics. You've got to say "from now on, I will do 25", and always do 25.

Hard, I know, but far easier than calculating the stats for the "I'll stop when I decide to on-the-fly" case. The stats ABX shows as you go along are based on the "pre-chosen fixed number of trials" model. The stats for "I'll stop when I decide to on-the-fly" are much stricter and require many many more trials.


Getting opinions from other people would be very valuable. They'd ultimately have to pass the ABX test too to be considered reliable. Otherwise they're just agreeing with you! Single blind tests (you know what track you're playing, but they don't) do not count as "reliable". It's probably something to try first, but it's not proof.


Thanks for taking the time to do this. I know that it is hugely time consuming!

Cheers,
David.

lossyWAV 1.2.0 Development Thread

Reply #245
BORK,

Regarding run length, just pick a single number and stick to it. Even saying "it will be between 20 and 40" breaks the statistics. You've got to say "from now on, I will do 25", and always do 25.

Hard, I know, but far easier than calculating the stats for the "I'll stop when I decide to on-the-fly" case. The stats ABX shows as you go along are based on the "pre-chosen fixed number of trials" model. The stats for "I'll stop when I decide to on-the-fly" are much stricter and require many many more trials.


Getting opinions from other people would be very valuable. They'd ultimately have to pass the ABX test too to be considered reliable. Otherwise they're just agreeing with you! Single blind tests (you know what track you're playing, but they don't) do not count as "reliable". It's probably something to try first, but it's not proof.


Thanks for taking the time to do this. I know that it is hugely time consuming!

Cheers,
David.


Thanks , will stick to a single value for runs, I agree that stats gathering would be way more effective when the same run times are used for all sessions , & even better for all testers, if there will be such an agreed upon number I will adjust to it.

Nick C.
I am going to do all the test files to the new version & parameters,
please tell me what command line to use for it.

lossyWAV 1.2.0 Development Thread

Reply #246
I'm not sure what quality level / shaping level you want to test....

"--extreme --shaping 0" would be a sensible start to see if there is anything definitive.

--extreme equates to --quality 7.5, so if you easily ABX --extreme then you can gradually increase the quality level without jumping straight to --insane (--quality 10.0).

At this time, I would suggest that you may also want the processing to be influenced more by higher frequencies. With this in mind, you could add --limit 17500 to the command line for a second run if the first is a simple ABX.

My guess is that you can hear sounds quite some way up the frequency scale....
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.2.0 Development Thread

Reply #247
I'm not sure what quality level / shaping level you want to test....

"--extreme --shaping 0" would be a sensible start to see if there is anything definitive.

--extreme equates to --quality 7.5, so if you easily ABX --extreme then you can gradually increase the quality level without jumping straight to --insane (--quality 10.0).

At this time, I would suggest that you may also want the processing to be influenced more by higher frequencies. With this in mind, you could add --limit 17500 to the command line for a second run if the first is a simple ABX.

My guess is that you can hear sounds quite some way up the frequency scale....


Nick, the reason Im asking is Id like to work on a Test File Set for all testers, (I will upload it), & using a single testing method , we can gather data you can analyze the stats for all submissions as testers join ,so I am assuming this will only have some value if we all use a single mode & method.

Only you can pick the one that should be the most urgent to gather stats for.

lossyWAV 1.2.0 Development Thread

Reply #248
Hmmm.... Since no-one previously had successfully ABX'd --standard then I think that that would be the starting point - with no modification to any other settings.

If we get some feedback indicating that --standard can indeed be ABX'd then maybe --extreme would be the next step. I would think of this as a sort of binary search (0 to 10, pick 5; if 5 is ABX'd then 5 to 10, pick 7.5; etc.) to determine the transparency point (if any).

There has been some talk regarding the noise-shaping implementation. I believe that this is pretty much boxed off in that the blips seem to have largely been removed in 1.1.2f onwards.

lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848 --scale 0.5 | FLAC -5 -e -p -b 512 -P=4096 -S- (having set foobar to output 24-bit PCM; scaling by 0.5 gives the ANS headroom to work)

lossyWAV 1.2.0 Development Thread

Reply #249
If you want lots of people to test, I think you'd better have examples starting from portable, or even lower!

You've got to be able to learn what you're listening for in an obvious way before making it less obvious.

(I suspect what BORK is hearing is quite different, but we'll have to start this in a conventional way at least).

Cheers,
David.