HydrogenAudio

Hydrogenaudio Forum => Listening Tests => Topic started by: C.R.Helmrich on 2010-01-06 22:29:47

Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-01-06 22:29:47
This week I started going through the vast amount of listening test material available (or linked to) in this forum. I noticed that many of the critical items recently used in ABX tests can be categorized as "transient (non-stationary) noise". The items I found which can be assigend to this category are:

- Amnesia (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=70598&view=findpost&p=639781)
- Autechre (http://www.hydrogenaudio.org/forums/index.php?showtopic=70442) (Artefact+Context version in 01__DCT_Killer_Samples__Lossless_.zip), also ae_gg (http://www.hydrogenaudio.org/forums/index.php?showtopic=49601) from the same artist
- Closer to God (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=70598&view=findpost&p=652197)
- Convulsion (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=70598&view=findpost&p=632363)
- Emese (http://www.hydrogenaudio.org/forums/index.php?showtopic=73970)
- Harlem (http://www.hydrogenaudio.org/forums/index.php?showtopic=70442)
- Human Disease (http://www.hydrogenaudio.org/forums/index.php?showtopic=67882)
- NiN Quake OST (http://www.hydrogenaudio.org/forums/index.php?showtopic=49601)
- 2nd_vent_clip.flac and drone_short from http://ff123.net/samples/ (http://ff123.net/samples/).

Any important ones I missed?

Now, with the public test coming up, I propose we use only two of those items in the test. As the first item, I propose the Harlem sample, since that's a natural sound (applause recorded at a live performance of the Harlem Gospel Singers). All the other items are more ore less artificial sounds and as such, do not represent typical listening material. Now, which one should we take as the second item? Should we make a poll?

Chris
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-01-07 00:37:26
I propose to use online randomizer (http://www.randomizer.org/links.htm) to select samples. Than the list of  numbers of random  samples will be sent automatically to all interested people who want to be sure that's selection was totally random.

Let's collect all samples from HA members and upload section and then public it. Samples from different styles and/or with different issues (not specifically for one competitor in particular).

I would avoid some of the samples here like "Closer to God" because they were submitted as Nero's issues only.
Title: Public Listening Test [2010], Item Selection
Post by: halb27 on 2010-01-18 21:59:32
I can't make a suggestion for transient noise samples but I agree very much to do include non-artificial samples.

I'm more into non-transient noise.
I suggest to use harp40_1 as a serious harpsichord problem (maybe it falls into the transient noise problem class partially - in this case we have another non-artificial sample for it).
From mp3 experience problems that strike me most are samples which are hard to beleive that they bring problems to encoders.
trumpet_MyPrince is such a sample which tends to get encoded with kind of a tremolo effect. An even worse sample for this encoding problem - of artificial nature - is lead-voice.
The very first problem sample I ran upon trumpet is prone to distortion at rather low bitrate.
herding_calls is also a tonal problem sample at least to low/moderate bitrate mp3.
I once ran upon subtle issues on a very simple song Là Ou Je Suis Née when using mp3.

The samples are here:
harp40_1 (http://dl.getdropbox.com/u/2681777/ProblemSamples/harp40_1.flac)
trumpet_My Prince (http://dl.getdropbox.com/u/2681777/ProblemSamples/trumpet_MyPrince.flac)
lead-voice (http://dl.getdropbox.com/u/2681777/ProblemSamples/lead-voice.flac)
trumpet (http://dl.getdropbox.com/u/2681777/ProblemSamples/trumpet.flac)
Là Ou Je Suis Née (http://dl.getdropbox.com/u/2681777/ProblemSamples/Là%20Ou%20Je%20Suis%20Née.flac)
Title: Public Listening Test [2010], Item Selection
Post by: halb27 on 2010-01-18 22:17:21
I just looked up my lossyWAV problem sample collection and found a transient noise prone sample keys_1644ds (http://dl.getdropbox.com/u/2681777/ProblemSamples/keys_1644ds.flac).
Not just music, but non-artificial.

Oh, I forgot the link for herding_calls (http://dl.getdropbox.com/u/2681777/ProblemSamples/herding_calls.flac) in my last post.
Title: Public Listening Test [2010], Item Selection
Post by: /mnt on 2010-01-18 22:58:34
I find the ptp_robocop, musique_non_stop and the_robots samples, to be suitable for the test. I have tried them with Nero at q 0.40 and QuickTime TVBR at 60, and they can be ABXed effortlessly.

Suggested samples:
- Linchpin (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=4937)
- The Robots (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5655)
- Musique Non Stop (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5656)
- Show Me Your Spine (3:00 - 3:30 Edit) (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5657)
- Human Disease (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=4794)
- Hexonxonx (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5593)
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-01-18 23:11:21
Yep, linchpin has presented issues for all AAC encoders in my personal test.

Samples from previous public test and from well know http://ff123.net/samples.html (http://ff123.net/samples.html) to avoid biased decisions.
Title: Public Listening Test [2010], Item Selection
Post by: /mnt on 2010-01-18 23:20:30
Yep, linchpin has presented issues for all AAC encoders in my personal test.


It sure is noticable issue with Nero AAC and LAME V5 - V0 and a big one for Helix Mp3. I haven't tried it on iTunes or QuickTime in a while though.
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-01-18 23:26:22
Vorbis and Apple LC-AAC performed very bad on linchpin at 80 kbps http://www.hydrogenaudio.org/forums/index....c=74781&hl= (http://www.hydrogenaudio.org/forums/index.php?showtopic=74781&hl=)
Title: Public Listening Test [2010], Item Selection
Post by: /mnt on 2010-01-18 23:46:00
Vorbis and Apple LC-AAC performed very bad on linchpin at 80 kbps http://www.hydrogenaudio.org/forums/index....c=74781&hl= (http://www.hydrogenaudio.org/forums/index.php?showtopic=74781&hl=)

I see, am suprised that Vorbis struggled with it aswell. For some reason this sample does sound better with LAME at 130kbps, even if i can ABX it at V2.

The Spill The Blood sample sounds bad on my cheap Logitech PC speakers.

QuickTime True VBR at 60 is just as bad:

Code: [Select]
foo_abx 1.3.4 report
foobar2000 v1.0
2010/01/18 23:35:06

File A: C:\Temp\128 Test\Linchpin.m4a
File B: E:\Music\Lossless\Fear Factory - Digimortal\05. Linchpin.flac

23:35:06 : Test started.
23:35:22 : 01/01  50.0%
23:35:30 : 02/02  25.0%
23:35:40 : 03/03  12.5%
23:35:59 : 04/04  6.3%
23:36:20 : 05/05  3.1%
23:36:29 : 06/06  1.6%
23:36:40 : 07/07  0.8%
23:36:47 : 08/08  0.4%
23:36:56 : 09/09  0.2%
23:37:08 : 10/10  0.1%
23:37:20 : 11/11  0.0%
23:37:29 : 12/12  0.0%
23:37:30 : Test finished.

 ----------
Total: 12/12 (0.0%)
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-01-19 00:45:59
Nice work, guys! Yes, the Linchpin sample today was promoted to test sample for tuning Fraunhofer's encoder

An addition to the "tonal items" halb27 uploaded: About half a year ago, I downloaded a sample called 06___Mandylion_shrt.flac from HA. It was supposedly uploaded by shadowking, as a quick search indicates. Can anyone post the link to it here? I don't want to upload it again because it's probably still available.

Chris

P.S.: It seems the item used to be in this thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=73344)? Where did it go?
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-01-19 02:30:26
Here are more killer samples:
Creuza (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=41248&view=findpost&p=362944) (tonality issue)
Spill the blood (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=3453). I submitted this quiet sample as issue for LAME -V5... V0 sometimes ago. One of my most tested samples
Waiting (http://ff123.net/samples/Waiting.flac) was tested in every public test.
Fatboy (http://www.hydrogenaudio.org/forums/index.php?showtopic=19682). MP3 encoders perform very bad on it (see last public MP3 test at 128). Would be interesting to see what happens with AAC encoders.
Aquatisme (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5658) from 48 kbps AAC test (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=49937&view=findpost&p=447579)
Herbie Hancock (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5659) (old record,jazz sample+ stereo separation test) was submitted sometimes ago as Apple issue but it's actually very hard sample for many codecs.
Descending Darkness (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5081). Very loud and reach drums content

These samples are the most interesting for me.
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-01-19 03:47:09
Links for some samples were broken.
http://www.hydrogenaudio.org/forums/index....showtopic=77994 (http://www.hydrogenaudio.org/forums/index.php?showtopic=77994)
Title: Public Listening Test [2010], Item Selection
Post by: muaddib on 2010-01-19 11:23:36
Why not taking in consideration organizing a database of samples as I've proposed here:

http://www.hydrogenaudio.org/forums/index....showtopic=64991 (http://www.hydrogenaudio.org/forums/index.php?showtopic=64991)

and then use public lottery for random choice of samples (also described in the thread)?
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-01-21 02:29:50
It's good idea.
My plan is a little bit limited to this particular test. The same methodology but only for 128 kbps. The organization of test is already time intensive.
The collaboration is welcome if someone else will take care of it.
Title: Public Listening Test [2010], Item Selection
Post by: hellokeith on 2010-01-21 06:43:29
Have all the item selections been already decided? I have a 10-20 second section of a track that I use which features some hi-hat hits quickly moving through the stereo field, very easy to identify if something is wrong.
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-01-21 08:54:49
Not at all! If you think it's a critical sample, please upload it in this thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=77994).

Thanks!

Chris
Title: Public Listening Test [2010], Item Selection
Post by: muaddib on 2010-01-21 19:47:19
It's good idea.
My plan is a little bit limited to this particular test. The same methodology but only for 128 kbps. The organization of test is already time intensive.
The collaboration is welcome if someone else will take care of it.

We could start creating the database that I propose from the samples that are prepared for this aac test. 3a, 3b, 4 and 5 subsets are adequate for this.
Maybe in the upload thread for the current test, a sticky could be created where each proposed sample is given number and where it is decided in which of subsets 3a, 3b, 4 or 5 it should go to.
Creating database during the preparation of the test is contradictory to what I wrote in the thread for database in post #8, but I guess that there is no other way to start organizing an "objective" database for listening tests.
Title: Public Listening Test [2010], Item Selection
Post by: rpp3po on 2010-02-15 21:47:04
[a href='index.php?showtopic=73970']Bibilolo[/a] hasn't been mentioned, yet, but should be in the selection.
Title: Public Listening Test [2010], Item Selection
Post by: rpp3po on 2010-02-19 16:00:50
I think we should also include a high quality applause sample with a nice stereo image. Any recommendations?
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-02-19 19:49:21
I think we should also include a high quality applause sample with a nice stereo image. Any recommendations?

See the first post.

From the link in Igor's post (#6) (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681482) I suggest "ATrain" and "SinceAlways". In that selection, there is also an "applaud00" (from an Eagles live recording), but that is so loud that it clips at several passages.

Chris
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-02-20 07:27:24
Took a look at http://www.listening-tests.info/mp3-128-1/results.htm (http://www.listening-tests.info/mp3-128-1/results.htm) before it closes down. Final Fantasy, Chariots Of Fire, Tom's Diner, and Velvet are a problem for at least one coder in the test. Final Fantasy we don't need in this test because we already have the SQAM harpsichord item (halb27, post 3 above (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681449)). Chariots Of Fire I can't find on HA, but I got a CD which contains it, so I can upload a critical passage if necessary. Tom's Diner is available in sample16.zip on http://www.mp3-tech.org/tests/aac_48/samples/ (http://www.mp3-tech.org/tests/aac_48/samples/). By the way, quite a few of the other samples suggested are available in ZIP files there. Velvet and some other potential candidates are at http://lame.sourceforge.net/download/samples/ (http://lame.sourceforge.net/download/samples/).

If time permits, I'll go through the last two linked collections to see if there are other samples worth including.

Chris
Title: Public Listening Test [2010], Item Selection
Post by: lvqcl on 2010-02-20 08:25:29
Quote
Chariots Of Fire I can't find on HA

http://www.hydrogenaudio.org/forums/index....st&p=601657 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=67529&view=findpost&p=601657)
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-02-20 08:58:17
Wonderful, thanks a lot! Somehow missed that thread 

Chris
Title: Public Listening Test [2010], Item Selection
Post by: hellokeith on 2010-02-21 05:25:46
Sample to test encoder stereo imaging transparency:

http://www.hydrogenaudio.org/forums/index....showtopic=78889 (http://www.hydrogenaudio.org/forums/index.php?showtopic=78889)
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-03-09 21:47:44
Due to absence of random selection mechanism and time limits it will be more realizable if Chris will be the person who will choose the samples with public agreement. Some most discussed samples are already included like Linchpin.
Personally I don't have doubts about Chris.
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-13 14:12:11
OK, guys, I need help here. The number of critical items I can dig up is overwhelming. So, in order of appearance:

halb27, post #3 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681449):

Your harpsichord 40 item is accepted for testing. Other items you proposed are from "mp3 times" and might not be AAC-critical. They are:
-  trumpet_My Prince
-  lead-voice
-  trumpet
-  Là Ou Je Suis Née
-  keys_1644ds
-  herding_calls

/mnt, post #5 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681472):

over the last few months you proposed a nice long list of AAC-critical items. Linchpin is already accepted for testing. Your remaining proposals are:
-  Kraftwerk remasters (the Zip file with excerpts you uploaded around Christmas?),
-  Show Me Your Spine (any 15-second passage you favor)
-  Human Disease
-  Hexonxonx
-  smothered_hope

IgorC, post #11 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681517):

many of the items you proposed to me are actually already in Fraunhofer's internal test set, so well known to me. The ones which are not are:
-  Creuza
-  Spill the blood
-  Aquatisme from 48 kbps AAC test
-  Descending Darkness
-  Girl
-  Erase_replace

To all above members:

Could you please ABX-HR your respective items list using the newest nero 1.5.4 -q 0.41 and QT True VBR Q60 and then report to me via personal message

-  which item is easiest to ABX vs. the lossless reference, regardless of coder (or in other words, averaged over the two coders)
-  how large the differences are for each item between the two coders without mentioning the coders (i. e. "huge differences", "both sound bad", etc.)

That would be a great help! Thanks,

Chris
Title: Public Listening Test [2010], Item Selection
Post by: loophole on 2010-03-13 16:28:29
It seems to me this more a test of how different encoders react to various killer samples - which is interesting but far less useful than a general audio quality test. You should test a wide selection of musical genres, not statistical anomalies (which all codecs will have) which have no bearing on overall sound quality whatsoever. It seems like an enormous waste of effort, have I missed something crucial here?
Title: Public Listening Test [2010], Item Selection
Post by: lvqcl on 2010-03-13 16:35:08
Quote
have I missed something crucial here?


This post (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77272&view=findpost&p=687652) and below.
Title: Public Listening Test [2010], Item Selection
Post by: sauvage78 on 2010-03-13 17:02:00
For me the idea that codecs have an "overall general quality" is a joke, it's either you can tell that there is a difference & describe how it sound different to you, or you can't.

Before I did some ABXing for myself & wasted several days to understand how lossy music sounded different from lossless music, I used to speak with term like this "overall general quality" ... nowaday I think that this way of speaking is only due to a lack of understanding of what you are speaking about: for me in the mouth of a newbie: "overall general quality"=placebo.

The only "overall general quality" of a codec I know is mix between:
1: how many killer samples affect the codec.
2: how bad is the distortion that you can hear within the killer samples that affect the codec.

What is a killer sample? It is a sample that you can ABX, simply. There is no "overall general quality" outside of killer samples because you cannot evaluate the quality of sound that you cannot even ABX.

This is specially true at medium/high bitrate because then generic music (not selected) will be transparent 99% of time ... evaluating the quality of transparent samples is non-sense.

I agree that you can speak of "overall general quality" of a codec at low bitrate (strictly below 128Kbps for modern encoders) because at say 96Kbps even random (non-killer) music is likely to not be transparent. Then yes "overall general quality" can have some meanings, but IMHO that is a special case.

I know that the idea that such thing as an "overall general quality" exist is due to people speaking of codecs in a very generic way like "this codec sounds metallic", with modern codecs this is definitly a word abuse & a generalization.

Don't be fooled by the langage, if a codec pass killer samples you can rest in peace: generic music will be an health walk for the codec.

Edit: I disagree with Gurubolez's above opinion, but that's also why I value personnal test that I can reproduce like those of /mnt more than public listening tests. /mnt killer samples are often gold to me.
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-13 17:39:16
Edit: I disagree with Gurubolez's above opinion, but that's also why I value personnal test that I can reproduce like those of /mnt more than public listening tests. /mnt killer samples are often gold to me.

Which is why I like to include some of /mnt's samples in this public test  By the way, do you have some AAC-critical samples of your own to add to this list?

And: which of guruboolez' posts are you referring to?

Chris
Title: Public Listening Test [2010], Item Selection
Post by: sauvage78 on 2010-03-13 18:06:48
1: No, I don't have personnal samples for AAC, the simple reason being that I actually don't use lossy at all. I like Nero AAC LC's quality (Specially 0.55) but I have a personnal problem with AAC not being gapless natively (from MPEG specifications). Stealing other ABXer samples is the reason why I read this topic  I keep an eye on AAC 's quality discussion because I might use it one day with video, but it is unlikely that I will ever use it for music. I always buy new HDD to keep using lossless.

2: Those linked within lvqcl' post.
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-13 18:20:45
1: I like Nero AAC LC's quality (Specially 0.55) but I have a personnal problem with AAC not being gapless natively (from MPEG specifications).

True, the lack of gapless playback standardization is shocking even for me. But there might be some progress on this subject soon.

Chris
Title: Public Listening Test [2010], Item Selection
Post by: googlebot on 2010-03-13 19:00:25
Since all thinkable gapless implementations need the same metadata (encoder delay, actual number of samples) I wouldn't worry too much about it. Apple's implementation is the de-facto standard right now. And if there is every any future formal specification, it will be trivial to copy the existing iTunes metadata into a tag compatible to the new scheme.
Title: Public Listening Test [2010], Item Selection
Post by: sauvage78 on 2010-03-14 00:36:30
Well, the following is about gaps, I know it is offtopic here but I don't know where to answer:

I did a very short & very simple test after googlebot's post, I encoded my lossless Pink Floyd - 1973 - The Dark Side Of The Moon rip to Nero AAC V1.5.4.0/quality 0.55 with F2K V1.01 & I listened between track 5 & 6 to see if I could hear anything bad, either added silence or a glitch.

Code: [Select]
  TRACK 05 AUDIO
    TITLE "Money"
    PERFORMER "Pink Floyd"
    INDEX 01 19:24:35
  TRACK 06 AUDIO
    TITLE "Us And Them"
    PERFORMER "Pink Floyd"
    INDEX 01 25:56:35


With the tags created during the encoding I couldn't hear any glitch.

Now I deleted the tags with Mp3tag v0.45a & re-listened to the transition between track 5 & track 6, & guess what ... now without tags there is an audible glitch ... I don't know if this is an issue with Mp3tag but all I know is that this very simple test (It takes less than 5 min) shows that losing the gapless playback metadata information by misstake is actually very easy with the actual Nero trick ... so as long as a simple tag edition will end in the possibility of losing gapless playback personnaly I will not use AAC for music. (I may use it with video as gaps are not an issue there)

Even if there is a standard for this one day & even if it is a trivial task to convert the actual metadata trick to this future standard ... it is actually so easy to lose this information that you may have lost it before a more robust standard exist.

My dream lossy codec is an MPEG ISO standard codec that achieve the quality of Nero AAC, with a native gapless support as good as Vorbis/Musepack (gapless directly in the specification). The actual tag trick is not satisfying for me. In the future this issue may lead me to re-use Vorbis while I know from my listening tests that Nero AAC beats Vorbis qualitywise. Actually I only use lossless in order to avoid choosing between plague & cholera ...
Title: Public Listening Test [2010], Item Selection
Post by: googlebot on 2010-03-14 02:26:36
I don't understand the issue. Proper gapless tags were saved. You removed them with a tagging tool, that didn't honor them. And gapless info was lost. The situation would not be different if the tags were written in a to be proposed ISO format, as long as the tool you are using doesn't honor them.

PS I just realized: sorry for the ongoing off-topic debate. Feel free to remove it from the thread.
Title: Public Listening Test [2010], Item Selection
Post by: sauvage78 on 2010-03-14 02:43:04
Well indeed I obviously deleted manually the metadata tags but the issue is that you cannot delete them as easily with a simple tag editor from other codecs with native gapless support. So on the one hand many people (incl. newbies) edit their tags, on the other hand very few people (advanced users) are aware of the gap problem untill they suddenly hear a glitch. This disproportion leads to the conclusion that many unaware AAC users might sooner or later lose their gapless metadata which is "volatile". Indeed it is not really an issue for people like us who know which misstake not to commit in order to keep their metadata, but it is not natural for beginners to think that editing tags can hurt the playback of their files. This tricky tag solution is just not friendly to newbies & not fully satisfaying IMHO. Feel free to disagree that's just my opinion.

Edit:
Even if from a technical point of view the info wouldn't be very different if it was in the specification, the metadata would be buried deeper in the files. Embedded inside, instead of wrapped around. Undeletable & supported by default by all decoders. For me gapless playback is not something "optionnal", it should be at the heart of any modern codec. Despite its great audio quality, with regard to gaps, AAC is not really a modern codec. It seems to me that nothing evolved for gaps between mp3 & AAC, which means that so far all audio codecs designed by MPEG are thought for video users & not audiophiles.
Title: Public Listening Test [2010], Item Selection
Post by: TechVsLife on 2010-03-14 04:16:10
Did you report this bug to mp3tag?  (I assume this doesn't occur when editing tags with itunes.)  I also have some issues with the itunes aac tag format, including the lack of support for multiple items in a tag (such as multiple artists), but it is the standard now, and I doubt another (lossy) format will succeed it.  (had it arisen now, the tagging format chosen would probably have been xml.)


Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-14 18:34:05
1: No, I don't have personnal samples for AAC, the simple reason being that I actually don't use lossy at all.

But here (http://www.hydrogenaudio.org/forums/index.php?showtopic=70442&view=findpost&p=622411) (very helpful test, by the way!), you mentioned a Ginnungagap item. Is that AAC-critical? If yes, can you point us to that one, or upload it here (http://www.hydrogenaudio.org/forums/index.php?showtopic=77994)?

Thanks,

Chris
Title: Public Listening Test [2010], Item Selection
Post by: sauvage78 on 2010-03-14 19:12:17
Ginnungagap (a sample from a Therion song) is a very noisy critical item for lossywav, I haven't seriously tested it with AAC but I doubt it would be as critical as the encoding technique used are very different. I honestly don't recall that I tested it with classic lossy encoder but it is likely that I quickly did but found nothing & that it's why I give up the idea of cross testing lossywav killer samples with classic lossy encoders ... but at the same time testing plenty of DCT killer samples on lossywav I found that Abfahrt Hinwil & Fool's Garden samples (which were originally found with classic lossy encoders) were also critical for lossywav ... so there is definitely the possibility that lossywav killer samples affects DCT codecs. But for Ginnungagap I honestly don't know, it was found by Martel on lossywav & has remained a lossywav specific test sample. It could be tested but as with all listening test it takes time. I am not sure it is really worth it because even if I think that this sample might be hard to encode at 96Kbps, it has very few chance to be as interesting as Harlem & Autechre at 128Kbps.
Title: Public Listening Test [2010], Item Selection
Post by: sauvage78 on 2010-03-15 00:14:31
TechVsLife:
I just tested the broken gapless playback problem with Mp3tag V2.46 in case it was already fixed, sadly the problem is still here, so I sended a PM with a link to Post #34 to Florian.

Edit: For fun I tested with aoTuVb5.7 ... with or without tags no glitch indeed. It shows that audio quality/compression is not everything, features are very important too.
Title: Public Listening Test [2010], Item Selection
Post by: TechVsLife on 2010-03-15 01:15:16
@sauvage78: thanks.  unlike the ms/apple bureaucracy, florian does correct bugs quickly (otoh, ms/apple have infinite wealth and life).
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-19 22:37:29
For public (and personal) reference, some more potential items from LAME 3.96 tuning days (some have already been mentioned here):

http://www.hydrogenaudio.org/forums/index....showtopic=19882 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882)

I also re-uploaded the abovementioned Mandylion item in our accompanying upload thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=77994).

Chris
Title: Public Listening Test [2010], Item Selection
Post by: hellokeith on 2010-03-20 10:00:50
It seems to me this more a test of how different encoders react to various killer samples - which is interesting but far less useful than a general audio quality test. You should test a wide selection of musical genres, not statistical anomalies (which all codecs will have) which have no bearing on overall sound quality whatsoever. It seems like an enormous waste of effort, have I missed something crucial here?


If the "killer samples" are from readily available music, they are valid.

If the "killer samples" are very rare one-off's like some guy playing three notes on a violin in his bathroom, they are invalid.

Unless I misunderstood this test was not about music?
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-20 10:37:08
I don't want to repeat myself so much, so please start with this (posts #195-200) (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77272&view=findpost&p=687680).

Who says that killer samples must stem from readily available music? They must be readily available (CD or download), but not necessarily what we consider music.

Plus, some musical pieces (e.g. Jazz) have isolated instruments in them, so it's not that far off. And guess why we have these items in our list - because solo instruments can reveal artifacts that spectrally complex music can't reveal.

Plus plus, only half of the samples used in MPEG tests are musical pieces. The rest are instruments from the SQAM CD, speech recordings, etc.

Chris
Title: Public Listening Test [2010], Item Selection
Post by: hellokeith on 2010-03-21 22:34:27
2. Samples.
Different styles of music, different levels of difficulty, pointing issues etc....?  To be discussed here or in separate topic.

Chris,
I was working from the above, so I apologize for missing that the sample selection had migrated away from a broad range.  While I agree that 128kbps in modern codecs is substantially better than it was 5 years ago, I don't agree that 128kbps is too high for a valid test based on a broad range of samples including mainstream music.  I'll have to go along with guruboolez that with a test based on rare extreme samples, my interest is nil.
Title: Public Listening Test [2010], Item Selection
Post by: googlebot on 2010-03-21 23:13:06
As far as I understand, the result for a "broad range" 128kbit/s test can be known in advance "mostly transparent with few exceptions". That's niler than nil.
Title: Public Listening Test [2010], Item Selection
Post by: TechVsLife on 2010-03-22 01:06:44
Who says that killer samples must stem from readily available music? They must be readily available (CD or download), but not necessarily what we consider music.

I'm a layman, but fwiw, I'm somewhere in the middle of the two camps: I think there is value in going for killer or extreme samples, because some people are using lossy formats as their only formats, i.e. as formats for both casual portable and more demanding at-home listening, so there's a need to test the possibility of glaring artifacts (since they have no pure archive format as a backup).  (Also that might affect individual decisions about one's need for lossless fornats.)   

However, I don't see a purpose for picking non-music or super-artificial samples EXCEPT and INSOFAR as they help developers fine tune their encoders for non-artificial music samples. 

Lossy compression is always lossy with a purpose; otherwise, you couldn't decide which bits you can afford to lose or not.  I don't think lossy music compression should be construed as SOUND compression, i.e. faithful reproduction of any sound or noise, because I assume that would defeat some of the techniques used which depend on harmonics and because the reason that people use mp3/m4a compression is for the appreciation of music and songs, and not for copying arbitrary sounds.  Even speech compression is better served by speech-specific codecs. 

To me, extreme metal and synthetic music starts to enter a non-music world; that may be taste only and others might want a codec geared precisely to reproduce that genre very faithfully.  But if there has to be a tradeoff between faithful reproducing of "natural" music and say the most extreme amusical/synthetic samples, I'd say there's great justification for gearing it to the faithful reproduction of music (or if you will traditional music).  --Of course, if there's no tradeoff, there's no problem here, but I assume that there's some correlation between the character of the killer samples we see and the difficulty of encoding them.  Also a caveat, there's some traditional music that is difficult to encode (harpsichords) so it is somehow "unnatural" from the point of view of the encoder (--if the encoder had a musical taste).

I assume the lossy encoders can to some extent "know" what genre they are facing, speech, heavy metal, etc. and switch techniques accordingly, though apparently none of them are good enough to use all the bits they need for the killer samples, perhaps because of the constraints imposed by some common lossy techniques that work so well for 99% of music.
Title: Public Listening Test [2010], Item Selection
Post by: googlebot on 2010-03-22 03:06:47
I assume the lossy encoders can to some extent "know" what genre they are facing, speech, heavy metal, etc. and switch techniques accordingly, though apparently none of them are good enough to use all the bits they need for the killer samples, perhaps because of the constraints imposed by some common lossy techniques that work so well for 99% of music.


That's usually not happening. Every sample faces the same psy model. It will have a different states as a function of the sample's context, but that's all. There is usually no higher level mode-switching or genre detection going on. A local context is all that's needed for the encoder to make optimal decisions. 2-pass encoding is an exception, but that doesn't do anything, that could be called "genre detection", either.

And it's not a bug, it's a feature. The code would be a mess to maintain and tuning a generalized solution makes much more sense. It would be different, for example if one encoder for both guitar and avantgardistic electronic music would be a contradicting design goal, but it's not. The design goal is exploiting your ear and auditory system, for whatever one throws at it.

I would agree to not include any synthetic samples, that were especially created to fuck a specific implementation. With enough knowledge about piece of code, there will probably always be some way to exploit it. But that's not true for any of the submitted samples, as far as I can see. Some of them may sound strange to your ears, but it is an encoder's job to fool them anyway and be transparent.

And LAME, Fraunhofer, Nero, and Apple have become so good, that I really don't see much sense for yet another listening test, that turns out as transparent for item 1-14, almost transparent for item 14-15. We know that already and 1-14 will be a major pain to test, which would frustrate many potential listeners. So why not go for hard nuts only, and see which encoder can crack the most.
Title: Public Listening Test [2010], Item Selection
Post by: TechVsLife on 2010-03-22 05:09:28
A local context is all that's needed for the encoder to make optimal decisions. 2-pass encoding is an exception, but that doesn't do anything, that could be called "genre detection", either.

Thank you, I don't know how the encoders do their magic.  At least with the local context, it looks like they still "know" to throw more bits at certain things and not at others, which end up corresponding to higher bitrates on e.g. heavy metal music.  ok, not genre detection at all, but a detection of local difficulty/hardness that if sustained over the piece can perhaps correspond in some way to genre.  I also didn't know the same rules were used, switching to more bits with the same rules depending on the local context.

. It would be different, for example if one encoder for both guitar and avantgardistic electronic music would be a contradicting design goal, but it's not. The design goal is exploiting your ear and auditory system, for whatever one throws at it.

Here I'm not sure I agree.  If it turns out that encoding efficiently for avantgardistic electronic music (lowest bitrate, highest quality) produces worse quality than encoding efficiently for guitar music, then it seems there wouldn't in fact be one design goal (i.e. as a practical matter), regardless of intention.  Now that, I take it, is not the case.  But it could be the case, e.g. some sounds are not easily reduced to a score (sheet music)--"sheet music" encoding hasn't been designed efficiently for reproducing all sounds (likewise midi).  Some compressors work better on text formats, some on binaries and jpegs.  Even though we would prefer there to be one compressor that worked best for all, we are forced to make a choice (or the compression engine decides on the fly).  It could be that certain kinds of sounds are more efficiently compressed by one kind of encoder than another.
Title: Public Listening Test [2010], Item Selection
Post by: googlebot on 2010-03-22 12:04:56
Some compressors work better on text formats, some on binaries and jpegs.  Even though we would prefer there to be one compressor that worked best for all, we are forced to make a choice (or the compression engine decides on the fly).


That's a completely different case than (perceptual) compression of audio. Binary (non audio) data can have regularity (and also complexity) several magnitudes larger than usual audio data. Binary (non audio) data can have repetitive patterns spanning larger ranges than what could be found by brute force search. So it can make sense to tune a lossless data compressor for specifically structured patterns out of the infinite problem space. With the exception of looped, electronic music the same is not usually true for audio.

There are also perceptual encoders tuned for special use cases, like speech codecs. They have some modification above what a general purpose encoder would do on the fly. But their goal isn't necessarily transparency, but sounding good enough at very low rates*. For general purpose encoders, which target transparency, there is no "on the fly" mode switching. All it does, is look at how many bits it does still have available at the current position and what its estimate for the audibility of each component in the current window are.

* Especially lower sample rates, for which usual block sizes are insufficient.
Title: Public Listening Test [2010], Item Selection
Post by: TechVsLife on 2010-03-22 15:06:04
That's a completely different case than (perceptual) compression of audio. Binary (non audio) data can have regularity (and also complexity) several magnitudes larger than usual audio data. Binary (non audio) data can have repetitive patterns spanning larger ranges than what could be found by brute force search. So it can make sense to tune a lossless data compressor for specifically structured patterns out of the infinite problem space. With the exception of looped, electronic music the same is not usually true for audio.


That's interesting; I hadn't realized audio data is less regular than most non-audio data files, though I can see the structured patterns in non-audio data could be more varied or complex, since they are not limited by the constraints that exist for music (rules of harmony, types of instruments, mechanics of vibration and hearing and medium). But I would have thought that music also can have "repetitive patterns spanning larger ranges than can be found by brute force search." Perhaps that's not something perceptual audio encoders can capture (but certainly evident in the sheet music of, say, a Bach cantata), since compressors are looking for simple or low kinds of patterns--can't compress by "understanding," only by math on patterns of numbers though that math might reflect or result from higher patterns.

I still don't see why it might not turn out that the types of patterns in one kind of music could in principle be more efficiently compressed using one method than those in another, even though non-audio data has as you observe more regularity. (Also, couldn't lossy encoders enhance the possibility of finding regularity, drawing them closer to non-audio data, since you have discretion to discard some of the data and smooth things out a bit.) (btw, don't know what "infinite problem space" is. and maybe you're using "complexity" in a particular technical sense.)
Title: Public Listening Test [2010], Item Selection
Post by: googlebot on 2010-03-22 15:14:54
You cannot search long ranges in polynomial time for arbitrary, repetitive patterns. But this is getting too OT. Feel free to open a separate thread.
Title: Public Listening Test [2010], Item Selection
Post by: TechVsLife on 2010-03-22 20:19:12
you're right it's too OT so I'll stop here.  (Looks like it might require having some detailed technical knowledge.)
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-23 00:05:07
OK, the list of test items is almost ready. Details on each item will follow later.

The crosses indicate the type of artifacts that can predominantly occur. Note that depending on the encoder, there might be other artifacts in the decoded samples than the ones mentioned in the table.

Code: [Select]
.________________________________________________________________________
|                 |                  |                  |                |
| Test Sample     |    Transients    |   Stereo Image   |    Tonality    |
|                 | Lo-Freq  Hi-Freq | Direction  Width | Noise  Tremolo |
|_________________|__________________|__________________|________________|
|                 |                  |                  |                |
| Kalifornia      |    +        +    |                  |                |
|                 |                  |                  |                |
| Robots          |    +             |                  |           +    |
|                 |                  |                  |                |
| Applause/Harlem |             +    |              +   |                |
|                 |                  |                  |                |
| Fallen/Linchpin |             +    |              +   |                |
|                 |                  |                  |                |
| Can't Wait      |                  |              +   |                |
|                 |                  |                  |                |
| Memories        |                  |              +   |                |
|                 |                  |                  |                |
| Waiting         |                  |              +   |                |
|                 |                  |                  |                |
| Since Always    |                  |     +            |                |
|                 |                  |                  |                |
| Hancock         |                  |     +            |                |
|                 |                  |                  |                |
| Berlin Drug     |                  |                  |   +            |
|                 |                  |                  |                |
| Trumpet/Rhumba  |                  |                  |   +            |
|                 |                  |                  |                |
| Creuza          |                  |                  |           +    |
|                 |                  |                  |                |
| Girl            |                  |                  |           +    |
|                 |                  |                  |                |
| Metropolis      |             +    |                  |           +    |
|                 |                  |                  |                |
| SQAM Selection  |             +    |                  |   +       +    |
|                 |                  |                  |                |
| (Velvet,spahm)  |                  |                  |                |
|_________________|__________________|__________________|________________|


It's a total of 16 items, 15 seconds each. Makes exactly 4 minutes of material. Velvet/spahm are still open for discussion, since I don't know what to put here. It doesn't have to be one of those two, though, they are just a suggestion. Does anyone know an item not mentioned yet on which both nero and QT/iTunes do really bad at 128kb?

Edit: Let's put a deadline for the final decision on item #16 (Velvet,spahm): April 1, 2010

Chris
Title: Public Listening Test [2010], Item Selection
Post by: googlebot on 2010-03-23 00:22:43
I'm astounded what  elaborate contributions this test gets.
Title: Public Listening Test [2010], Item Selection
Post by: /mnt on 2010-03-23 00:45:43
Does anyone know an item not mentioned yet on which both nero and QT/iTunes do really bad at 128kb?


Human Disease and Techno Pop sounds pretty bad at 128kbps, on iTunes, QuickTime and Nero AAC.
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-23 21:44:08
Sorry, I should document better my motives for the above item selection. In the course of this discussion, I was convinced that "rare extreme", "artificial" samples should be excluded since they are indeed quite far from typical musical genres such as pop, rock, and electro. Hence, I excluded items such as emese, bibilolo, lead-voice, aquatisme, all the ones in post #1 of this thread, and many of /mnt's "zappy" items (sorry, but I hope you agree that such sounds are extreme, rare, and unusually artificial). There is one exception: the two Kraftwerk items because those also test for tonality and low-frequency transient artifacts. In addition, I believe that something like "some guy playing three notes on a violin in his bathroom" is a valid, natural critical sample, so I included a few of those (note that such samples are still the minority).

I'm aware that emese, bibilolo, etc. are "classic" killer samples, so if someone wants to do a separate test on such items after this test, I'm happy to help and listen.

Conclusion: To anyone having proposals for item #16 (Velvet,spahm), please make your suggestions accordingly.

Thanks,

Chris
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-03-24 03:14:49
Chris,
Can you please upload listed samples in .rar somewhere?
Thank you.
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-24 23:39:28
Not yet, since I haven't actually constructed the samples. But I can give you the references to each item used in the samples. Sorry such details trickle in slowly, but that's all the time I have per day.

Kalifornia: first 14.6 seconds of Fatboy:
http://www.hydrogenaudio.org/forums/index....st&p=681517 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681517)

Robots: cuts from beginning and mid part of the Robots remaster:
http://www.hydrogenaudio.org/forums/index....st&p=695016 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=695016)
Maybe this will be concatenated with a similar item if I can find one.

Applause/Harlem: concatenation of applaud00 and Harlem:
http://www.hydrogenaudio.org/forums/index....st&p=688933 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=688933)
http://www.hydrogenaudio.org/forums/index....st&p=621624 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=70442&view=findpost&p=621624)

Fallen (should be Fall)/Linchpin: concatenation of Fall of Life and Linchpin:
http://www.hydrogenaudio.org/forums/index....st&p=195618 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=19882&view=findpost&p=195618)
http://www.hydrogenaudio.org/forums/index....st&p=682220 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=682220)

Can't Wait: 15-second passage from Can't Wait until Tonight:
http://www.hydrogenaudio.org/forums/index....st&p=195478 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=19882&view=findpost&p=195478)

Memories: 15-second passage from Cats musical:
http://www.hydrogenaudio.org/forums/index....st&p=692203 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=692203)

Waiting: first 15 seconds from Green Day's song:
http://www.hydrogenaudio.org/forums/index....st&p=681517 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=681517)

Since Always: first 15 seconds from that song:
http://www.hydrogenaudio.org/forums/index....st&p=688933 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=688933)

Hancock: excerpt from a Jazz piece by Herbie Hancock:
http://www.hydrogenaudio.org/forums/index....st&p=681520 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=681520)
Maybe this will be concatenated with a similar item (if I can find one) since it is shorter than 15 seconds.

Berlin Drug: excerpt from an unknown song (does anyone have more info on this?)
http://www.hydrogenaudio.org/forums/index....st&p=682234 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=682234)

Trumpet/Rumba: concatenation of trumpet and an excerpt of Entierren con Rumba:
http://www.hydrogenaudio.org/forums/index....st&p=682125 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=682125)
http://www.hydrogenaudio.org/forums/index....st&p=201619 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=19882&view=findpost&p=201619)

Chris
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-25 19:48:10
Continued:

Creuza: first 15 seconds of Creuza de mä:
http://www.hydrogenaudio.org/forums/index....st&p=362944 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=41248&view=findpost&p=362944)

Girl: first 15 seconds of that song (Igor, what's the full name, artist, and origin?):
http://www.hydrogenaudio.org/forums/index....st&p=683378 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77994&view=findpost&p=683378)

Metropolis: first 15 seconds of Kraftwerk's song from the Man Machine, again the remaster:
was from http://www.hydrogenaudio.org/forums/index....showtopic=77128 (http://www.hydrogenaudio.org/forums/index.php?showtopic=77128),
I don't want to re-upload it, it will become available when I present the finished test samples, anyway.

SQAM Selection: a concatenation of items from the Sound Quality Assessment Material CD released in 1988 by the European Broadcasting Union:
http://tech.ebu.ch/publications/sqamcd (http://tech.ebu.ch/publications/sqamcd)
This collection is widely used in the scientific literature, so why not include it here. It's now available free of charge for research and development purposes! The tracks used will be 32 (triangle), 35 (glockenspiel), and 40 (harpsichord).

--

I also found another candidate for item #16: The 15-second passage starting at 7.0 sec of Flyin' to Fly:
http://www.hydrogenaudio.org/forums/index....st&p=363343 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=41248&view=findpost&p=363343)
It's quite a general-purpose critical sample, tests SBR (which we don't have here), tonality, and pre-echo issues.

For completeness:
"Roel's infamous Velvet (http://lame.sourceforge.net/quality.php)" and spahm: http://lame.sourceforge.net/download/samples/ (http://lame.sourceforge.net/download/samples/)

Chris
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-03-27 16:33:08
Song Title: Girl
Artist: Beck
Album: Guero (2005)
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-03-28 00:12:41
Vocal sample with very rich frequency content.
roter sand (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5809)

Chris, how many samples will be included? 16-17? 16-18 is fine.
Title: Public Listening Test [2010], Item Selection
Post by: C.R.Helmrich on 2010-03-28 17:50:29
I was aiming for 16, like I wrote in post #15 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77584&view=findpost&p=695576). IMO, even that is already on the high side, and more would be too tiring for the listeners.

Chris
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-04-01 21:23:19
I've tried Velvet and Fly to Fly sample. They aren't  killer samples for me.
First sample from http://www.hydrogenaudio.org/forums/index....showtopic=75868 (http://www.hydrogenaudio.org/forums/index.php?showtopic=75868)
presents problem at least for 2 AAC encoders.
Title: Public Listening Test [2010], Item Selection
Post by: udauda on 2010-04-01 22:36:52
SQAM Selection: a concatenation of items from the Sound Quality Assessment Material CD released in 1988 by the European Broadcasting Union:
http://tech.ebu.ch/publications/sqamcd (http://tech.ebu.ch/publications/sqamcd)
This collection is widely used in the scientific literature, so why not include it here. It's now available free of charge for research and development purposes! The tracks used will be 32 (triangle), 35 (glockenspiel), and 40 (harpsichord).


There's a companion document (http://tech.ebu.ch/publications/tech3253) that goes along with the SQAM disc. EBU truly kicks a$$ for releasing this gem to the public.
Title: Public Listening Test [2010], Item Selection
Post by: Sylph on 2010-04-02 00:56:32
So if all goes as planned when will this test begin and how long will it last?
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-04-02 03:44:44
Now we're almost done with selection of samples it's time to prepare packages, web page and start test.
Title: Public Listening Test [2010], Item Selection
Post by: IgorC on 2010-04-04 21:55:13
"Ectasy" is a killer sample for all AAC encoders at 128 kbps.
http://www.hydrogenaudio.org/forums/index....st&p=590404 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=64410&view=findpost&p=590404)

Ecstasy (http://www.hydrogenaudio.org/forums/index.php?act=attach&type=post&id=5834)
Title: Public Listening Test [2010], Item Selection
Post by: TechVsLife on 2010-07-05 21:15:15
Has this test started?
Title: Public Listening Test [2010], Item Selection
Post by: muaddib on 2010-07-06 09:25:51
Too bad that there is nobody at the moment with enough interest and free time to organize the test.
We miss you Sebastian and Roberto