Skip to main content

Topic: "near transparent" listening test (Read 6979 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • elmar3rd
  • [*][*][*]
"near transparent" listening test
Are the samples and the ini/batch-files for ABC/HR tool in the 64 kbps-test a good starting point for a quick listening test in the "near transparent" range, too?

This test schould not work out the best "near transparent" codec/setting, but how the common and easy available codecs perform in comparison.

IMHO it's useful and interesting to examine thesis like "Regarding hardware compatibility, LAME APS ist still a good choice".

Codecs/Settings for example:
- OGG Vorbis q6 and/or Ogg Vorbis GT3 q5
- LAME APS
- MMJ fastenc 192
- Musepack quality 5
- Nero ACC and/or QT AAC
- WMA9
  • Last Edit: 24 August, 2003, 05:41:52 AM by elmar3rd

  • tigre
  • [*][*][*][*][*]
"near transparent" listening test
Reply #1
This is a nice idea but it won't work IMO for many reasons:

- At "near transparent" settings you need a lot of training, patience, time etc. to perform a test like this. I doubt that there'd be enough ppl participating and providing useful results

- All rating would be somthing close to 5, so after statistical evaluation there's probably nothing useful left as codecs are simply too close.

- With most samples of the 64kbps test most codecs won't fail, so probably > 90% can't be ABXed and get a 5 rating.

- Choosing known problem samples for this kind of test isn't an option either IMO as most problem samples are specific for one codec while others perform much better or have no problems at all.
  • Last Edit: 24 August, 2003, 05:52:28 AM by tigre
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

  • music_man_mpc
  • [*][*][*][*][*]
  • Members (Donating)
"near transparent" listening test
Reply #2
Quote
- MMJ fastenc 192

What the hell is that??  and how is it going to compete if it is 192 ABR or CBR, which is how it seems to be presented.
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame

  • bubka
  • [*][*][*]
"near transparent" listening test
Reply #3
this is the test i would be really interested in, something we all use
Chaintech AV-710

  • DeeZi
  • [*]
"near transparent" listening test
Reply #4
I think this test would be very interresting, especially the aac- and wma 9-part.

  • phong
  • [*][*][*][*]
"near transparent" listening test
Reply #5
The test would have to be performed with problem samples.  Normal samples are transparent even to golden ears at those settings and most problem samples tend to be difficult at best.  Further, while some things have a standard setting for "transparent" (mpc --standard, lame --alt-preset standard), other codecs do not.  Should we consider oggenc -q5?  I've found that oggenc -q6 is more often than not, bigger than lame -aps, even though vorbis is supposed to be better than mp3.

Also, if known problem samples are used, the test will be certainly skewed against codecs that are used around here most often.  In addition, the results won't be that meaningful because each codec is at a different bitrate.  Lets say, hypothetically (I doubt this would occur), that oggenc -q6 beats mppenc --standard.  That's meaningless because the vorbis files are much bigger.  There's no way to know, from these results, which would perform better at the same bitrate.

I think a test at 160kbps vbr (or something similar) would be better.  You'd still have a problem of finding suitable samples and suitable listeners, and the skew that results from having only problem samples.
I am *expanding!*  It is so much *squishy* to *smell* you!  *Campers* are the best!  I have *anticipation* and then what?  Better parties in *the middle* for sure.
http://www.phong.org/

  • music_man_mpc
  • [*][*][*][*][*]
  • Members (Donating)
"near transparent" listening test
Reply #6
Quote
The test would have to be performed with problem samples.  Normal samples are transparent even to golden ears at those settings . . .

Thats not what this thread:  http://www.hydrogenaudio.org/forums/index....=0&#entry122726 would seem to suggest.

edit: not that I'm saying this test is workable.  guruboolez, geko and trnsz would likely be the only ones with useful input . . . . but you never know I'm thinking about trying to train my ears, just to see if I have what it takes.
  • Last Edit: 24 August, 2003, 03:15:22 PM by music_man_mpc
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame

  • tigre
  • [*][*][*][*][*]
"near transparent" listening test
Reply #7
Quote
I'm thinking about trying to train my ears, just to see if I have what it takes.

Think twice before you start!    I have done too much testing/ABXing during the last 2 weeks (trying to find good samples for 64kbps test and http://www.soundexpert.info/). Now some of my favourite downloaded mp3s (some of them 224kbps) sound annoying to me and I had to buy the CDs (hard to find). 
  • Last Edit: 24 August, 2003, 03:53:47 PM by tigre
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

  • mdmuir
  • [*][*][*]
  • Members (Donating)
"near transparent" listening test
Reply #8
As a side note I think a lot of how one reads the results of all these abx tests is still irrecoverably linked to what one perceives the usefulness of the results.

To clarify-when Roberto was asking for samples for his 64 kbps test, I decided to do some abx tests of four of the codecs involved (vorbis, wma9pro, HE AAC , mp3pro) with just music I like, not problem samples. My results showed me that wma9pro was the least objectionable to MY ears, but not by much. And I would never choose 64 kbps encoding for my personal needs.

So, when the results of Roberto's 64 kbps tests are in-my reading of it will be not
"which codec is the best for 64kbps encoding" but rather "which codec is the least crappy at 64 kbps". This is my admitted built in bias against using anything with 64kbps encoding. And such bias would immediately preclude me from being a tester in the first place.

This is why I am so glad I chose to store all my music on computer losslessly-even though 99% of the time I could never abx it against 'transparent" lossy encoder  files, just knowing they are just a compressed version of the wav means I don't have to think or worry about it-just re-encode for whatever situation demands something different-ie aps for an mp3 portable cd player from the lossless files.
  • Last Edit: 24 August, 2003, 04:28:34 PM by mdmuir
you will make mp3's for compatibility reasons.

  • ScorLibran
  • [*][*][*][*][*]
  • Banned
"near transparent" listening test
Reply #9
Quote
Quote
I'm thinking about trying to train my ears, just to see if I have what it takes.

Think twice before you start!    I have done too much testing/ABXing during the last 2 weeks (trying to find good samples for 64kbps test and http://www.soundexpert.info/). Now some of my favourite downloaded mp3s (some of them 224kbps) sound annoying to me and I had to buy the CDs (hard to find). 

This is exactly the way I feel, too!  I just (a few days ago) started encoding my collection into Ogg Vorbis Post 1.0 CVS -q 4.25 because I did some ABX tests on some of my music (about 20 samples), and couldn't differentiate it even once from source PCM WAV files.  As a bonus, I can now fit my entire 357 album (402 CD) collection onto a 20GB HDD.  Before, I was using FLAC, and could only fit about 60 CDs in the same space.  I'm not sure whether to feel proud or embarrassed to have found a 136kbps nominal bitrate as my preferred format.  And actually, though 136kbps is the "stated" nominal bitrate, I'm getting more like a 126-127kbps average overall (about 0.93 MB/min measured after having encoded about 100 CDs).

I'm astounded that I could be truly happy with this -q level, and I'm afraid if I start "training" my ears, then I'll start hearing artifacts in my music that I didn't hear before.  The primary reason I didn't participate in Roberto's 128 kbps listening tests is that I had only marginal-quality equipment...Yamaha AC-XG sound adapter, Sony MDR-ED228 earbuds.  Now that I have better hardware (Echo Indigo, Xin headphone amp, Grado SR-80s), I'm still reluctant to participate in any arranged listening test.  To feel of any value in doing so, I'd first want to spend time training my ears to hear different artifact types.  After I was done, I could probably participate in a valuable way in a formal listening test, but after it was all done, I'd have to encode all my music again into FLAC!  Why couldn't I just step up the -q level a notch or two?  Because one of my primary listening platforms (car) can't currently decode anything higher than -q 4.25 Vorbis (1.0), so to go any higher, it's LAME MP3 --aps...gappy, so that's out.  Next step is FLAC.      (Doesn't speak MPC.)

Anyway, it's great in a way that a <136kbps (nominal) encoding level is transparent to my ears for the bulk of my music.  It's an odd, different kind of Warm Fuzzy Feeling©, plus not needing to upgrade my 20 GB discs quite so soon is nice too.  At this point, I don't want to start messing with my level of audio perception...it's right where I want it! 

  • phong
  • [*][*][*][*]
"near transparent" listening test
Reply #10
Quote
Thats not what this thread: http://www.hydrogenaudio.org/forums/index....=0&#entry122726 would seem to suggest.

Well, that thread was really about one problem sample that Guruboolez dug up, and about a general class of problem samples for MPC.  I don't think it reflected on general transparency of MPC at --standard.  As I mentioned in that thread, I was able to find a problem sample (much easier to ABX than Guruboolez's sample) for lame --alt-preset insane and oggenc -q6 on my first try (sample here.)  I was not able to do the same (so far) for MPC.

Also, there's the problem mentioned in that thread and again here - training yourself to hear artifacts is a curse in a way.  Participating in the 128kbps test was my training, and I almost wish I could go back.  Ignorance really is bliss.  There's a track on an album I own (Fibber Island off of They Might Be Giants - No!) that was previously distributed as a 128kbps mp3 on eMusic.  When I was listening to my CDs looking for samples to volunteer for the 64kbps test, I heard what sounded like significant pre-echo in some guitar at the beginning.  This is a real pressed CD I bought from the store.  I'm fairly convinced now that they pressed the mp3ed version onto the CD.  Here's a sample.  I hope somebody can tell me that I'm crazy.  Edit: it turns out (to my relief) that I'm crazy.

It used to be that 128kbps mp3s bothered me, but now they're really offensive to my ears.  I can't even listen to them and wonder how I ever could.  I heard a friend's XM radio the other day and gagged.  It sounded worse than a FM station does (at least, when you've got good reception).  It was muffled (sounded like an 11khz lowpass or lower) and there were significant artifacts.  Turns out, all the stations are 64kbps or sometimes LOWER!  I can't believe how people rave about the quality (stuff like "It's way better than 160k mp3s").  Sure, there weren't commercials, and the reception was always good, but I can't believe people pay like 200 bucks for the reciever and then 10 bucks a month.
  • Last Edit: 25 August, 2003, 10:18:59 AM by phong
I am *expanding!*  It is so much *squishy* to *smell* you!  *Campers* are the best!  I have *anticipation* and then what?  Better parties in *the middle* for sure.
http://www.phong.org/

  • rjamorim
  • [*][*][*][*][*]
"near transparent" listening test
Reply #11
Quote
Well, that thread was really about one problem sample that Guruboolez dug up, and about a general class of problem samples for MPC.

Besides, you must take in consideration that guruboolez's hearing is supernatural. :B

I'm actually afraid he's one of those Volcano guys, with weird pointed ears and all. That's why he can pick up every damn artifact.
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • den
  • [*][*][*][*]
"near transparent" listening test
Reply #12
Quote
Also, there's the problem mentioned in that thread and again here - training yourself to hear artifacts is a curse in a way.


Whenever these types of discussions come up, I am reminded of the following -

Quote
Neo: What truth?
Morpheus: That you are a slave, Neo. Like everyone else you were born into bondage, born into a prison that you cannot smell or taste or touch. A prison for your mind.... Unfortunately, no one can be told what the Matrix is. You have to see it for yourself. This is your last chance. After this there is no turning back. You take the blue pill, the story ends, you wake up in your bed and believe whatever you want to believe. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.... Remember, all I'm offering is the truth, nothing more.... Follow me....


and then

Quote
Cypher: You know, I know what you're thinking, because right now I'm thinking the same thing. Actually, I've been thinking it ever since I got here. Why, oh why didn't I take the blue pill?...


Why, oh why did I start reading HA, and started noticing all sorts of annoying artifacts in my tunes... 

  • ScorLibran
  • [*][*][*][*][*]
  • Banned
"near transparent" listening test
Reply #13
Quote
Quote
Cypher: You know, I know what you're thinking, because right now I'm thinking the same thing. Actually, I've been thinking it ever since I got here. Why, oh why didn't I take the blue pill?...


Why, oh why did I start reading HA, and started noticing all sorts of annoying artifacts in my tunes... 

la - la - la - la - la - :fingers in my ears: - la - la - la - la - la - la - la - la - la - la - :not listening: - la - la - la - la - la - :happy in my cell: - la - la - la - la - la - la - la - la - la - :the world's too big out there: - la - la - la - la - la - la - la - la - la - la - :but I don't wanna hear any artifacts: - la - la - la - la - :I'm hearing it the way the music's supposed to sound: - la - la - la - la - la - la - la - :I'm happy with it just the way it is: - la - la - la - la - :5000 songs on 20 GB: - la - la - la - la - la - la - :5000 songs on 20 GB: - la - la - la - la ...

....... must .......... not ................. hear ........................................ artifacts .....

..... too busy looking at the woman in the red dress ....

  • Gecko
  • [*][*][*][*][*]
"near transparent" listening test
Reply #14
Quote
la - la - la - la - la - :fingers in my ears: - la - la - la - la - la - la - la - la - la - la - :not listening: - la - la - la - la - la - ...

Maybe you should change your sig then. To "Viva status-quo!" maybe.

Anyway, back on topic. The 64k test samples weren't chosen to be easy to encode or anything. They are just interesting pieces of music from various genres which people here enjoy. Any real world music is suitable for a listening test. If they sound transparent then the codec has succeeded in its task and you can be happy to have such a wonderful encoder.

  • music_man_mpc
  • [*][*][*][*][*]
  • Members (Donating)
"near transparent" listening test
Reply #15
Quote
I'm actually afraid he's one of those Volcano guys, with weird pointed ears and all.

Do you mean a 'Vulcan' like Spock, from Star Trek??

edit: sorry about the off-topicness

edit: OK, lets make this a less off-topic post.  I already think that 128 mp3 sounds like shit, I can even ABX --alt-preset cbr 192 without too much trouble, but I can't COME CLOSE to being able to ABX MPC.  I can't even ABX waiting at --quality 4.
  • Last Edit: 25 August, 2003, 02:44:09 PM by music_man_mpc
gentoo ~amd64 + layman | ncmpcpp/mpd | wavpack + vorbis + lame