lossyWAV transcoding test discussion 2010-09-01 12:00:32 Quote from: carpman on 2010-08-17 16:35:33Just as an aside, I think a major transcode test of LossyWAV would be far more beneficial than the currently planned HA 2010 public test.I'm not volunteering to run it, but it might be time to consider such a test, if anyone is willing to run it. So I've started this thread to discuss it.Apart from carpman, does anyone else think something like this is worthwhile? Would you participate?These are my assumptions (please anyone feel free to jump in if you think they're wrong or you have different ideas or suggestions etc)...1. We'll have lossless original source, lossyWAV intermediate, lossy-from-lossless and lossy-from-lossyWAV.2. We'll ABX (i) lossless vs lossy-from-lossless, (ii) lossless vs lossy-from-lossyWAV, (iii) lossy-from-lossless vs lossy-from-lossyWAV, and also ask participants whether they think lossy-from-lossless or lossy-from-lossyWAV is closer to the lossless original (in the last task, you need to know which is the original. Everything else is double-blind).3. We should pick one version of lossyWAV, and one or two settings to test. e.g. standard and portable4. We should pick between one and three lossy formats as target. I think mp3 is essential, vorbis is desirable, and AAC is optional.5. We should use lossy bitrates that people actually use - e.g., for mp3, somewhere between lame V2 and V5.6. We should test samples that are most likely to be problem samples. These would probably be (A) known lossyWAV problems samples, (B) known problems samples for the lossy codecs under test, and © any that tick both boxesMy initial comment is that I really don't like the look of item 2. It looks like a lot of hard work for the test participants, and I'm not sure which software is appropriate to do it. Maybe we could skip (i) and (ii). Maybe there's a better test methodology altogether (e.g. MUSHRA?).Also, ambitious numbers of things to test under items 3-6 will easily lead to 100s of combinations (=tests) in total. This won't work. Need to focus, somehow.Cheers,David.