Proposition for database of listening test samples 2008-07-22 17:03:54 Please comment on this idea:We create online database of listening test samples. Then define some subsets:1a. Samples that must be tested in 24-48kbps1b. Other samples that are candidates for 24-48kbps test2a. Samples that must be tested in 56-98kbps2b. Other samples that are candidates for 56-96kbps test3a. Samples that must be tested in 112-320kbps3b. Other samples that are candidates for 112-320kbps test4. Cases specific for particular encoder/format5. Cases where some people hear problems even at high bitratesBitrates are only approximations and actually represent 3 levels of encoding quality (low bitrate, acceptable for portable, near transparent).We can also create subsets for 4 levels.When testing only one format we include samples that represent problem only for one format (from 4) in the corresponding 1a-3b. For each test case in 4 it is defined at which bitrate it starts to produce problems.Subsets 1a, 2a and 3a should have only small number of known problematic samples for the target bitrate and should always be included in a test for the target bitrate. They should represent problems for the bitrate, but not total misuse of low bitrate (e.g. fatboy should not be in 1a).Samples in subset 1b, 2b and 3b should represent wide variety of music (different genres).Some samples from subset 1b, 2b or 3b are chosen based on some public lottery numbers.Chosen samples together with corresponding 1a, 2a or 3a are then test samples.To which subset a sample belongs is to be decided based on previously conducted public listening tests.A sample can be in more than one subset, but subsets 1a and 1b, 2a and 2b, 3a and 3b don't have intersections.New samples can be added and a sample can change subset it belongs to. To add new sample, a person proposes one. Then it should be tested by at least 2 people using different encoders/formats.Based on the results it is decided to which subsets should it be added.A sample can also be removed from database if consensus is met that it is not interesting anymore.Subset 5 is specific and samples belonging to it can also be part of other subsets. 3a and 5 shouldn't have an intersection - because a sample from 5 can not be distinguished by most people at high bitrates.All samples should be very short (< 20s, preferably 8-15s). If previously used sample in a public listening test is longer, then it should be shortened (removing uninteresting parts) or split into 2 parts. Big number and wide variety of samples should compensate for short duration.Having short samples attracts more people to a listening test and makes them more concentrated.Also online space issue would be easier to deal with.Final decisions about a sample being added or removed will be problematic in some cases, and I hope we will find a solution for it. Maybe some board of well known HA members.