Skip to main content

Topic: lossyWAV transcoding test discussion (Read 6018 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
lossyWAV transcoding test discussion
Just as an aside, I think a major transcode test of LossyWAV would be far more beneficial than the currently planned HA 2010 public test.
I'm not volunteering to run it, but it might be time to consider such a test, if anyone is willing to run it. So I've started this thread to discuss it.

Apart from carpman, does anyone else think something like this is worthwhile? Would you participate?


These are my assumptions (please anyone feel free to jump in if you think they're wrong or you have different ideas or suggestions etc)...

1. We'll have lossless original source, lossyWAV intermediate, lossy-from-lossless and lossy-from-lossyWAV.
2. We'll ABX (i) lossless vs lossy-from-lossless, (ii) lossless vs lossy-from-lossyWAV, (iii) lossy-from-lossless vs lossy-from-lossyWAV, and also ask participants whether they think lossy-from-lossless or lossy-from-lossyWAV is closer to the lossless original (in the last task, you need to know which is the original. Everything else is double-blind).

3. We should pick one version of lossyWAV, and one or two settings to test. e.g. standard and portable

4. We should pick between one and three lossy formats as target. I think mp3 is essential, vorbis is desirable, and AAC is optional.
5. We should use lossy bitrates that people actually use - e.g., for mp3, somewhere between lame V2 and V5.

6. We should test samples that are most likely to be problem samples. These would probably be (A) known lossyWAV problems samples, (B) known problems samples for the lossy codecs under test, and © any that tick both boxes


My initial comment is that I really don't like the look of item 2. It looks like a lot of hard work for the test participants, and I'm not sure which software is appropriate to do it. Maybe we could skip (i) and (ii). Maybe there's a better test methodology altogether (e.g. MUSHRA?).

Also, ambitious numbers of things to test under items 3-6 will easily lead to 100s of combinations (=tests) in total. This won't work. Need to focus, somehow.

Cheers,
David.

lossyWAV transcoding test discussion
Reply #1
For a start I could add lossyWAV to SE rating system if nobody minds.
keeping audio clear together - soundexpert.org

  • pdq
  • [*][*][*][*][*]
lossyWAV transcoding test discussion
Reply #2
I wouldn't bother with portable. If you are planning to transcode then you should probably be using the higher quality setting.

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
lossyWAV transcoding test discussion
Reply #3
For a start I could add lossyWAV to SE rating system if nobody minds.
I'm happy. I suppose some recent version, using "portable", encoded to FLAC -8 (for your bitrate calculation), would make most sense. There's wavpack lossy too. Neither are psychoacoustic codecs though, so your "typical" problem samples may be unfair.

Cheers,
David.

  • Alex B
  • [*][*][*][*][*]
lossyWAV transcoding test discussion
Reply #4
I think it would good to start with a pretest that would give some experience of what to expect and the potential hazards.

Perhaps something like

encoder 1: lossless (reference)
encoder 2: lossless > LAME -V5
encoder 3: lossless > lossywav standard > LAME -V5
encoder 4: lossless > lossywav portable > LAME -V5
encoder 5: lossless > lossywav zero > LAME -V5 (low anchor)

The samples should mostly be from the category ( c ), if possible.

If it is labeled as a pretest then the presentation could be informal - perhaps just offer links to the lossless sample files and a write a how-to guide. The testers could post their results directly to the forum thread. If someone would like to test a different lossy encoder or setting that would be fine because we would get more useful information.

lossyWAV transcoding test discussion
Reply #5
I suppose some recent version, using "portable", encoded to FLAC -8 (for your bitrate calculation), would make most sense. There's wavpack lossy too. Neither are psychoacoustic codecs though, so your "typical" problem samples may be unfair.

Seems new section DSP would be more appropriate for such kind of processing. Realy "portable"? May be "standard"?
keeping audio clear together - soundexpert.org

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
lossyWAV transcoding test discussion
Reply #6
Realy "portable"? May be "standard"?
I don't think there's a single sample been ABXed at standard for a while now - could be wrong.

Cheers,
David.


  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
lossyWAV transcoding test discussion
Reply #7
@Alex B - excellent idea.

Anyone want to nominate sample(s)?

Cheers,
David.

lossyWAV transcoding test discussion
Reply #8
Realy "portable"? May be "standard"?
I don't think there's a single sample been ABXed at standard for a while now - could be wrong.

As we use artifacts amplification they will be clearly audible, but resulting rating could be pretty high. The question is how high? In comparison with 320 kbit/s encoders for example. In any case the audio metrics that we use is new and the ratings should be accepted with care.

So standard preset is OK. lossyWAV 1.2.0 is the latest I suppose. If so I'm ready to add it.
keeping audio clear together - soundexpert.org

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
lossyWAV transcoding test discussion
Reply #9
So standard preset is OK. lossyWAV 1.2.0 is the latest I suppose. If so I'm ready to add it.
Yes, that's a good choice. Nick's PM'd to say he's happy too.

Cheers,
David.

  • halb27
  • [*][*][*][*][*]
lossyWAV transcoding test discussion
Reply #10
For a start I could add lossyWAV to SE rating system if nobody minds.

I appreciate this very much.
lame3995o -Q1

lossyWAV transcoding test discussion
Reply #11
keeping audio clear together - soundexpert.org

  • Nick.C
  • [*][*][*][*][*]
  • Developer
lossyWAV transcoding test discussion
Reply #12
Thank you very much, Serge - much appreciated.

[edit] Sp. [/edit]
  • Last Edit: 07 September, 2010, 01:24:12 AM by Nick.C
lossyWAV -q X -a 4 -s h -A --feedback 2 --limit 15848| FLAC -5 -e -p -b 512 -P=4096 -S-