128kbps Extension Test - OPEN
Reply #172 – 2003-07-29 10:51:36
1/ bachpsicord should be discarded completely since it is a 'freak occurrence' that only shows an abnormal bitrate for one (or a few) codecs and will therefore color the test. Harpsichord is known to be one of the most difficult instrument for many lossy encoders, sensitive to pre-echo (sharp attack) and subjet of heavy distorsions during the development of the note. I see two advantages on it : - easy to ABX for many people (at least, easier than a piano or a violin) - interesting to see how will react a psymodel, and bitrate consequences (for exemple, mpc and vorbis will detect the difficulties, and increase the bitrate ; a VBR setting for lame, as --preset-standard, underate difficulties, and will produce an abnormal distorted sound) And harpsichord isn't a 'freak occurence'. It's just like saying that electric guitar is something rare, and should be removed : pure non-sense. Harpsichord was the most used instrument during at least two century, in the whole Europa. According to my listening tastes, harpsichord is more usual than any kind of cymbals. A listening test had to include a good variety of musical genre and various samples, in order to be 'fair' and representative.2/ bachpsichord is just a good example where a particular codec (or a few codecs) have trouble. Yes, exactly. But not for a single or even few codecs, but for most lossy & perceptual format. I know one exception : wma and wma9pro. I know wma to be bad on most other samples - critical or not. Don't know wma9pro general performance. Therefore, it's interseting to test various format on a difficult instrument ; mixing VBR and CBR will make thing more intersting (and certainly not unfair), because I know that one pure CBR encoder, at 128 kbps, may sound as good if not better than others at 160-200 kbps.But I think in the end we are bound by some 'luck' choosing the samples. It might be that Roberto has chosen totally horrible samples from mpc point of view and totally great for ogg.. there's no way to prevent such faults, really. (unless you start listening to complete music pieces). The best test would be to have one sample of every song ever made but that's kinda hard to do. If you assume that Roberto has chosen fairly 'average, representative' samples that represent a lot of the different possible challenges lossy codecs face, then a codec that has more punt-weighted bad problems should receive negative points, the 'representative' choice of -all- samples should cancel out the freak occurrences with the exeption samples. Did you ever made a listening test ? Did you ever took the responsabilty of choosing the adequate and representative samples, easy enough for being detected from original ? What made a sample 'exceptional', and which of the twelve is really 'freak' ? Did you participate to this test ?