Skip to main content

Topic: Perception and Measurement of Headphone Sound Quality: Do listeners ag (Read 8276 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • solive
  • [*][*][*]
Perception and Measurement of Headphone Sound Quality: Do listeners ag
The science behind the perception and measurement of headphone sound quality is not well understood. There are very few published studies based on controlled headphone listening tests, and the current diffuse-field and free-field standards for calibrating headphones that are perceived as too bright and/or thin in the bass. Today there is little consensus among headphone manufacturers on how a headphone should sound and measure. This is in stark contrast to other audio components and loudspeakers where most manufacturers at least agree the frequency response should be flat.

In the past 18 months, we've been conducting controlled listening experiments on headphones to understand if listeners agree on what makes a headphone sound good,and how to best measure headphones so that a perceptually meaningful measurement and design specification can be developed. Most recently, we've been testing different headphones and target responses using both trained and untrained listeners from different age groups and different cultures. In this way, we can study whether age, culture or listening experience influences preference in headphone sound quality.

A summary of this research can found in a presentation I recently gave at the ALMA 2014 Winter Symposium. I've provided a summary and PDF  of the presentation here:.

I'd be interested in hearing your feedback on this research. Thanks!

Cheers
Sean Olive
  • Last Edit: 08 January, 2014, 12:14:25 AM by solive
Sean Olive
Audio Musings

  • dhromed
  • [*][*][*][*][*]
Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #1
I'm getting a weird deja-vu. Haven't I seen this post and/or research before, here on HA, from you? Perhaps a previous version or variation of this research?

  • solive
  • [*][*][*]
Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #2
I'm getting a weird deja-vu. Haven't I seen this post and/or research before, here on HA, from you? Perhaps a previous version or variation of this research?


Someone previously started  a thread on the first paper we wrote back in October 2012 where we evaluated 6 different popular headphones using a method of substitution where the test administrator substituted the headphones from behind the listener. We found there was generally good agreement among trained listeners in what they preferred and this tended to correlate with their perceived spectral balance and measured response.
1. Sean E. Olive and Todd Welti, "The Relationship between Perception and Measurement of Headphone Sound Quality", presented at the 133rd Audio Eng. Soc. Convention, San Francisco, USA, (October 2012).


There have been 4 additional papers since then. The second paper we evaluated different headphone target response curves implemented over two different headphones  (Sennheiser HD518 and Audeze LCD 2). These unequalized headphones were compared against different versions of the diffuse and free-field calibrations and a new target response based on an accurate loudspeaker calibrated in a reference listening room. This new target response based on the in-room response of a loudspeaker in a room was preferred over both DF and FF targets, and the unequalized headphones.
2. Sean E. Olive, Todd Welti and Elisabeth McMullin, "Listener Preferences For Different Headphone Target Response Curves",  presented at the 134th Audio Eng. Soc. Convention, Budapest, Hungary, (May 2013).


In the third paper we developed and validated a virtual headphone listening test methodology where we simulate different headphones over a reference headphone by equalizing it to match the measured response of the target headphone. In this way, we can make A/B comparisons among different virtual headphones in a truly double-blind manner.
3. Sean E. Olive, Todd Welti and Elisabeth McMullin, "A Virtual Headphone Listening Test Methodology", presented at the 51st Audio Eng. Soc. International Conference, Helsinki, Finland, (August 2013).


In the fourth paper, we studied the preferred in-room target response of a high quality loudspeaker in a reference listening room  and compared it to the preferred target response of a headphone. Using a method of adjustment, we had listeners directly adjust the bass and treble level of both headphone and loudspeakers to their preferred levels.
4. Sean E. Olive, Todd Welti and Elisabeth McMullin, "Listener Preferences for In-Room Loudspeaker and Headphone Target Responses"  presented at the 135th Audio Eng. Soc. Convention, New York, USA, (October 2013).


In the 5th paper, I reported some over-the-ear and in-ear headphone preference ratings of college students versus Harman trained listeners using the virtualized headphone methodology described in paper 3.
5. Sean E. Olive, "Do college students prefer the same headphone sound quality as trained listeners?", presented at the 4th ISEAT, Shenzhen, China, (November 2013).


In the past 3 months, we've developed an iPad software application that allows us to do very controlled and efficient headphone listening tests in the field to a) measure listeners' preferred bass and treble headphone levels, and b) measure listener preference ratings of different over-the-ear headphone models including the new Harman target response, the Sennheiser HD800, the Audeze LCD2 and the Beats Studio. The purpose is to see what is the preferred headphone target response and whether it's different from factors such as  listener training, age and culture.  While these tests are ongoing in the presentation I give a preliminary report based on 219 listeners I've tested in Canada, USA, China and Germany.
  • Last Edit: 08 January, 2014, 09:41:27 AM by solive
Sean Olive
Audio Musings

  • Gecko
  • [*][*][*][*][*]
Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #3
I'm a bit surprised by the "Preferred Bass and Treble Levels". I always thought "flat = good" but maybe that's too universal. I've seen many people, who bother to measure and correct in-room response, target a similar "house curve" with increased bass and reduced treble. The Audyssey target curve also rolls-off the treble.

Would a speaker with a flat frequency response produce a similar in-room response as indicated by the "Preferred Bass and Treble Levels" in a typical living room (whatever that is)? I'm assuming that standing waves will probably increase perceived bass and directivity will lead to decreased treble?

I guess the basic question is: if I were buying speakers, should I actually look for ones with a flat response?

More on topic: it's interesting and reassuring to see, that there is a consensus about good sound across ages and nations.

  • Gearme
  • [*]
Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #4
In the 5th paper, I reported some over-the-ear and in-ear headphone preference ratings of college students versus Harman trained listeners using the virtualized headphone methodology described in paper 3.
5. Sean E. Olive, "Do college students prefer the same headphone sound quality as trained listeners?", presented at the 4th ISEAT, Shenzhen, China, (November 2013).

In the past 3 months, we've developed an iPad software application that allows us to do very controlled and efficient headphone listening tests in the field to a) measure listeners' preferred bass and treble headphone levels, and b) measure listener preference ratings of different over-the-ear headphone models including the new Harman target response, the Sennheiser HD800, the Audeze LCD2 and the Beats Studio. The purpose is to see what is the preferred headphone target response and whether it's different from factors such as  listener training, age and culture.  While these tests are ongoing in the presentation I give a preliminary report based on 219 listeners I've tested in Canada, USA, China and Germany.


Interesting research...a few questions come to mind:
- Did the students all listen to the same songs or bring their own music? 
- How does this choice impact the study?
- What are the versions of the headphones in the study (i.e. LCD RevX, Beats 201X, etc.)?

  • dhromed
  • [*][*][*][*][*]
Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #5
I'm assuming that standing waves will probably increase perceived bass


Standing waves will make the bass seem harsh and distorted, not simply increased. What I've experienced with any speakers in any room in my entire life, is that in a given ordinary room that isn't a special dead room, some small range of bass frequencies will always resonate with the room's size, cause overamplified standing waves and make music unlistenable after a relatively short time. Since that effect does not exist in headphones, it's easier to boost bass across the board and still have a pleasing sound that one may describe as thick, warm, full etc.

I do not like speakers for that reason. I well and truly despise subwoofers for that reason as well. Destroy them. Destroy!

  • KozmoNaut
  • [*][*][*][*][*]
Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #6
I guess the basic question is: if I were buying speakers, should I actually look for ones with a flat response?


I would say that yes, you should always look for a flat response in speakers. The reasoning here is that bass or treble or whichever EQ curve you prefer can easily be added to the signal path using tone controls or equalizing, but it's very hard to balance out a built-in non-flat response with those same tools.

I do not like speakers for that reason. I well and truly despise subwoofers for that reason as well. Destroy them. Destroy!


And yet, nothing gives music and movies that 'full' sound that a properly-built and properly-setup subwoofer does. Even something simple like a car door thunk in a movie or people running around on the decks overhead in Master and Commander takes on a fullness and realism that simply can't be replicated through headphones, because they lack the full-body physical feedback inherent to subbass sound. It is hugely ironic that the speaker(s) designed to only play a frequency range of 30-100hz at the very most must also be so much physically larger and more powerfully amplified than any other component, but you cannot make big deep sound without moving large amounts of air.

The best sound reproduction I have ever heard in my life was watching the aforementioned Master and Commander at a friend's house. At the time, he had a set of Beolab 5 speakers. For those not familiar with those, they are active 4-way speakers containing a down-firing 15" subwoofer each, built-in room correction (microphone and all) and 2500W of built-in class-D amplification. Not even in the best cinemas have I heard such precise, well-measured, loud and deep bass. To my ears, the sound was completely lacking in distortion or destructive room interferences and resonances, it was amazingly good.

So I think subwoofers definitely have their place, but they should be tuned and setup to "disappear", to extend the frequency response of the speakers without drawing attention to themselves. Boom-boom subs are a different matter of course, they should all be banned and destroyed.

My own modest system is of course nowhere near the level of those Beolab 5s, but I still consider it quite good, including the subwoofer, because I have spent the time to set it up as well as possible and because I know how to exercise restraint with the volume control. And obviously because I chose a really good subwoofer as a starting point, instead of something built to satisfy teenagers with a binary approach to volume controls.
  • Last Edit: 24 January, 2014, 08:17:54 AM by KozmoNaut

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #7
teens like good sound too?

someone should tell the recording industry.

(see umpteen other threads).


Thank you for sharing this fascinating research Sean, and for making it easily accessible.

Cheers,
David.

Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #8
I'm assuming that standing waves will probably increase perceived bass


Standing waves will make the bass seem harsh and distorted, not simply increased. What I've experienced with any speakers in any room in my entire life, is that in a given ordinary room that isn't a special dead room, some small range of bass frequencies will always resonate with the room's size, cause overamplified standing waves and make music unlistenable after a relatively short time. Since that effect does not exist in headphones, it's easier to boost bass across the board and still have a pleasing sound that one may describe as thick, warm, full etc.

I do not like speakers for that reason. I well and truly despise subwoofers for that reason as well. Destroy them. Destroy!


Seems like correcting the standing waves would be a more positive move.

IME Bass traps and multiple subwoofers work.

  • dhromed
  • [*][*][*][*][*]
Perception and Measurement of Headphone Sound Quality: Do listeners ag
Reply #9
And yet, nothing gives music and movies that 'full' sound that a properly-built and properly-setup subwoofer does.


I admit that I turn up the bass knob on my amp a bit when watching movies (with headphones).

Your friend's setup sound wonderful.

Seems like correcting the standing waves would be a more positive move.


That's true, but I prefer a sprinkle of hyperbole in my posts.