At the beginning of the week I tried to do my first own ABX test (flac vs mp3), testing how good halb's extension is at using --cvbr auto.
I have to say I only tested one music sample ([a href=\'index.php?act=findpost&pid=0\']Dear J[/a]), doing 12 tests per setting, don't know if it's enough. Tell me if I should do more next time.
I started using -V5 --cvbr auto going up to -V4 --cvbr auto and -V3 --cvbr auto.
-V5 --cvbr auto
19:11:54 : Test started.
19:12:15 : 01/01 50.0%
19:12:21 : 02/02 25.0%
19:12:27 : 03/03 12.5%
19:12:31 : 04/04 6.3%
19:12:38 : 05/05 3.1%
19:12:52 : 06/06 1.6%
19:12:58 : 07/07 0.8%
19:13:03 : 08/08 0.4%
19:13:09 : 09/09 0.2%
19:13:15 : 10/10 0.1%
19:13:22 : 11/11 0.0%
19:13:30 : 12/12 0.0%
19:13:31 : Test finished.
----------
Total: 12/12 (0.0%)
-V4 --cvbr auto
19:21:06 : Test started.
19:22:52 : 01/01 50.0%
19:23:27 : 02/02 25.0%
19:23:43 : 03/03 12.5%
19:23:59 : 04/04 6.3%
19:24:18 : 04/05 18.8%
19:24:30 : 05/06 10.9%
19:24:38 : 06/07 6.3%
19:25:07 : 07/08 3.5%
19:25:33 : 08/09 2.0%
19:25:46 : 09/10 1.1%
19:25:54 : 09/11 3.3%
19:26:13 : 10/12 1.9%
19:26:21 : Test finished.
----------
Total: 10/12 (1.9%)
-V3 --cvbr auto
19:27:16 : Test started.
19:28:22 : 00/01 100.0%
19:28:36 : 01/02 75.0%
19:29:16 : 02/03 50.0%
19:29:27 : 03/04 31.3%
19:29:36 : 03/05 50.0%
19:29:58 : 04/06 34.4%
19:31:39 : 04/07 50.0%
19:31:45 : 05/08 36.3%
19:31:52 : 06/09 25.4%
19:31:59 : 07/10 17.2%
19:32:18 : 08/11 11.3%
19:32:25 : 09/12 7.3%
19:32:30 : Test finished.
----------
Total: 9/12 (7.3%)
I think nothing special so far. Also I have to admit that -V3 was sometimes quite difficult for me to figure out.
Next step I got from -V2 up to -V0 comparing halb's extension with the offical alpha version.
lame 3.100a2 -V2
19:46:01 : Test started.
19:46:06 : 01/01 50.0%
19:46:11 : 02/02 25.0%
19:46:15 : 03/03 12.5%
19:46:22 : 04/04 6.3%
19:46:28 : 05/05 3.1%
19:46:38 : 06/06 1.6%
19:46:50 : 07/07 0.8%
19:46:55 : 07/08 3.5%
19:47:00 : 08/09 2.0%
19:47:04 : 09/10 1.1%
19:47:08 : 10/11 0.6%
19:47:13 : 11/12 0.3%
19:47:15 : Test finished.
----------
Total: 11/12 (0.3%)
lame 3100k -V2 --cvbr auto
19:33:29 : Test started.
19:34:07 : 01/01 50.0%
19:34:37 : 02/02 25.0%
19:34:54 : 03/03 12.5%
19:35:16 : 04/04 6.3%
19:35:29 : 05/05 3.1%
19:36:19 : 05/06 10.9%
19:36:53 : 06/07 6.3%
19:37:01 : 06/08 14.5%
19:37:43 : 07/09 9.0%
19:37:52 : 08/10 5.5%
19:38:06 : 09/11 3.3%
19:38:24 : 10/12 1.9%
19:38:26 : Test finished.
----------
Total: 10/12 (1.9%)
lame 3.100a2 -V1
23:45:35 : Test started.
23:45:40 : 00/01 100.0%
23:45:45 : 01/02 75.0%
23:45:50 : 01/03 87.5%
23:45:54 : 02/04 68.8%
23:45:59 : 03/05 50.0%
23:46:03 : 03/06 65.6%
23:46:08 : 04/07 50.0%
23:46:12 : 04/08 63.7%
23:46:43 : 05/09 50.0%
23:46:50 : 06/10 37.7%
23:47:09 : 07/11 27.4%
23:47:22 : 08/12 19.4%
23:47:25 : Test finished.
----------
Total: 8/12 (19.4%)
lame 3100k -V1 --cvbr auto
23:21:36 : Test started.
23:21:56 : 01/01 50.0%
23:22:04 : 02/02 25.0%
23:22:11 : 03/03 12.5%
23:22:34 : 04/04 6.3%
23:22:41 : 05/05 3.1%
23:22:48 : 05/06 10.9%
23:23:39 : 06/07 6.3%
23:23:57 : 07/08 3.5%
23:24:48 : 08/09 2.0%
23:24:56 : 09/10 1.1%
23:25:03 : 10/11 0.6%
23:26:22 : 11/12 0.3%
23:26:24 : Test finished.
----------
Total: 11/12 (0.3%)
lame 3.100a2 -V0
00:00:05 : Test started.
00:00:50 : 00/01 100.0%
00:01:00 : 01/02 75.0%
00:01:16 : 02/03 50.0%
00:02:02 : 02/04 68.8%
00:02:18 : 03/05 50.0%
00:02:43 : 04/06 34.4%
00:02:54 : 05/07 22.7%
00:03:50 : 06/08 14.5%
00:04:09 : 07/09 9.0%
00:04:53 : 08/10 5.5%
00:06:13 : 08/11 11.3%
00:06:56 : 09/12 7.3%
00:06:59 : Test finished.
----------
Total: 9/12 (7.3%)
lame 3100k -V0 --cvbr auto
23:35:30 : Test started.
23:35:34 : 01/01 50.0%
23:35:41 : 02/02 25.0%
23:35:46 : 03/03 12.5%
23:35:50 : 04/04 6.3%
23:36:01 : 04/05 18.8%
23:36:18 : 05/06 10.9%
23:36:45 : 06/07 6.3%
23:36:52 : 06/08 14.5%
23:37:05 : 07/09 9.0%
23:37:20 : 07/10 17.2%
23:37:36 : 08/11 11.3%
23:40:21 : 09/12 7.3%
23:40:24 : Test finished.
----------
Total: 9/12 (7.3%)
For some reason when using -V1 --cvbr auto it wasn't really difficult to figure out the difference between mp3 and flac. I found it more difficult using -V1 in 3.100a2.
I have to say under my normal listening surrounding I wouldn't be able to to tell the difference using -V2 --cvbr auto and higher. Honestly before I started my test I thought I wouldn't be able to hear any difference using -V2 or higher, so after the test I was a bit surprised.
I did another test using 3100k with -V0 --cvbr 9, out of curiosity.
19:38:38 : Test started.
19:39:19 : 01/01 50.0%
19:40:04 : 02/02 25.0%
19:41:39 : 03/03 12.5%
19:41:50 : 03/04 31.3%
19:43:10 : 03/05 50.0%
19:44:39 : 03/06 65.6%
19:45:41 : 04/07 50.0%
19:48:16 : 05/08 36.3%
19:48:32 : 06/09 25.4%
19:50:43 : 07/10 17.2%
19:52:14 : 08/11 11.3%
19:53:01 : 08/12 19.4%
19:53:06 : Test finished.
----------
Total: 8/12 (19.4%)
Did repeat the test three times, but everytime I got 7-8/12, also I have to say it was quite hard to do that, but it was possible.
If someone wants to try testing it too, the link is at the beginning (14 seconds). Little hint: listen carefully from 1-2sec - start by using V5