Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Ethan Winer's "AES Damn Lies Workshop" Video (Read 12846 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Ethan Winer's "AES Damn Lies Workshop" Video

http://www.youtube.com/watch?v=Zvireu2SGZM

From the video description:

"This is a video version of my workshop 'Lies, Damn Lies, and Audio Gear Specs' from the October 2013 AES show in New York City. Panelists are Ethan Winer, Mike Rivers, Scott Dorsey, and David Moran. This video explains how the fidelity of audio equipment is assessed and measured, and is an offshoot of my book The Audio Expert"


 

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #1
Thanks very much Tomas.

--Ethan
I believe in Truth, Justice, and the Scientific Method

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #2
On thing issue with home audio needing to be as transparent as possible, it is true that some audiophiles use tube amplifiers and other older technologies and they claim that it provides some sort of coloration which they refer to as "warmth". What they're actually hearing may be up for debate, but nevertheless it raises the issue of how to get better sound in the home and how that relates to what was created in the studio. I suspect that many audiophiles don't really ask the question directly - they just buy equipment on the basis of some difference in sound that it may or may not have and don't think in terms of accuracy.

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #3
Lots of great stuff in there - as usual, Ethan! I tend to think half the data is more pernicious than a reviewer giving two thumbs up . . .

However, isn't part of the problem here not only that “your 'pleasant' isn't my 'pleasant' ” but more interestingly that your measurements aren't my measurements? It can all be measured, but as you said a verdict is only credible if everything is measured.

It's tremendously unhelpful to think of an audio system in terms of boxes: they're literally components in one entity which - as you mention - includes the listening environment. Measurements made in the context of one system are only valid within the context of that system - which inevitably includes the measuring equipment itself.

Given the difficulty of measuring every system in every room with sufficient accuracy to be credible, it's perhaps no wonder that people rely on listening so much. Practically, is there really an alternative?

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #4
Lots of great stuff in there - as usual, Ethan! I tend to think half the data is more pernicious than a reviewer giving two thumbs up . . .

However, isn't part of the problem here not only that “your 'pleasant' isn't my 'pleasant' ” but more interestingly that your measurements aren't my measurements? It can all be measured, but as you said a verdict is only credible if everything is measured.

It's tremendously unhelpful to think of an audio system in terms of boxes: they're literally components in one entity which - as you mention - includes the listening environment. Measurements made in the context of one system are only valid within the context of that system - which inevitably includes the measuring equipment itself.

Given the difficulty of measuring every system in every room with sufficient accuracy to be credible, it's perhaps no wonder that people rely on listening so much. Practically, is there really an alternative?


Believe it or not the performance of audio gear can be separated from the performance of the room it is in. For example, power amps are wonderfully indifferent to the rooms they are in, and the good ones perform nearly idealy with just about any reasonable loudspeaker. Ditto for DACs, digital music players and AVRs.

Speaker sound is somewhat different but a combination of room treatments and system equalization can produce acceptable sound with a wide variety of speakers in a wide variety of rooms.

Also, an executive summary of a recent series of papers by Sean Olive of Harman is that on the average, very many people's acceptable is acceptable to many people.

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #5
Meh.  For some there is profit to be made by raising doubt about established means of taking measurements and suggesting that everything sounds different.

This is not to say listening is a bad thing, as your ears (and only your ears) should be the final arbiter.  Of course one cannot isolate what enters the ears when the eyes are allowed to participate.

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #6
Meh.  For some there is profit to be made by raising doubt about established means of taking measurements and suggesting that everything sounds different.

This is not to say listening is a bad thing, as your ears (and only your ears) should be the final arbiter.  Of course one cannot isolate what enters the ears when the eyes are allowed to participate.


Tricky, isn't it? Wouldn't it be great if you could press a button on a dilithium-powered Star Trek analyser and it all be done with.

Back in the real world, though . . . far too often, reviews fall into the trap of generalising about subjective impressions within the context of a specific system and, alarmingly frequently, with a scant variety of program material. Genuine neutrality is best defined by failing to impose a characteristic on a wide variety of systems when reproducing a wide variety of recordings: the ability to disappear - which includes the capacity to recreate the most challenging extremes of performance - the music that sorts the men from the boys. For instance, the ability of a full-range driver not to distort mid-range and treble frequencies during passages of rigorous LF thrashing.

Complete measurement tells us everything, but let's not fall victim to the same lazy thinking as manufacturers and dress up half the story as the whole truth. See Dunning-Kruger . . .

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #7
Quote
However, isn't part of the problem here not only that “your 'pleasant' isn't my 'pleasant'
The "traditional" goal of high fidelity is to faithfully reproduce the recording (or even the sound of a live performance in some cases).  Making the recording sound pleasant is generally the responsibility of the artist and producer.

And, let's say you find bass boost pleasant.  Wouldn't you still like to see accurate frequency response specifications/measurements from the manufacturer?

Quote
but more interestingly that your measurements aren't my measurements?
If you measure 10% distortion and I measure 0.01% distortion, one of us is doing something wrong!

Quote
It can all be measured, but as you said a verdict is only credible if everything is measured.
But "Everything", is a limited number of measurements.  (See Mr. Winer's "Audiophoolery" article.)

Quote
It's tremendously unhelpful to think of an audio system in terms of boxes:
But, it truly is a series of boxes/components (and processes).  And, this is how science is done...  It's what "analysis" is... Breaking a system or process into it's components to study each one independently.

Quote
Given the difficulty of measuring every system in every room with sufficient accuracy to be credible, it's perhaps no wonder that people rely on listening so much. Practically, is there really an alternative?
But listening to "every system in every room" is no easier.

HydrogenAudio is all about scientific listening tests and things/differences that we actually can hear.    But, measurements can be useful too.  If an amplifier is down 10dB at 5kHz, it's going to sound "dull" (probably a tube amplifier  ).  Or, if you take an amplifier that's flat to 20kHz and make a modification extending the response to 100kHz or 1MHz, that's not going to make any audible difference. 

And, if I must rely on others doing the listening/testing, I generally trust someone else's technical measurements more than someone else's listening tests.    In fact if you read audio equipment reviews, the listening test reports almost never use scientific terminology (noise, distortion, frequency response).  They use useless "audiophile" terms like "warmth" "veil", "clarity", etc.... Words that seem to tell you something about the sound, but really don't.

With listening tests, A/B or ABX (comparing to a reference) is almost a requirement for useful results.  That means you need a standard reference that everybody agrees on.  Even if we could agree (on the best reference loudspeaker, for example), someone would come-along with a new or better standard and we'd have to start-over.

With technical tests, you can make absolute measurements so there is no need for a reference system to compare to.  And, we only have to agree on the significance of the results.

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #8
The "traditional" goal of high fidelity is to faithfully reproduce the recording (or even the sound of a live performance in some cases).  Making the recording sound pleasant is generally the responsibility of the artist and producer. And, let's say you find bass boost pleasant.  Wouldn't you still like to see accurate frequency response specifications/measurements from the manufacturer?

Ethan's point was well made re: the job of the recording engineer to exercise an aesthetic judgment about sounding 'pleasant', but thereon down the chain, the game is solely fidelity of reproduction: no added 'character' is required - a bit like a professional kitchen! And the sharp practice in the audio industry re: published specs deserves to be put in the spotlight: I completely applaud that.

]If you measure 10% distortion and I measure 0.01% distortion, one of us is doing something wrong!

Not at all: if I measure 10% in my system and you measure 0.01% in yours, both results are valid - in fact, such field testing often reveals issues that were not present in lab conditions. Feedback.

But, it truly is a series of boxes/components (and processes).  And, this is how science is done...  It's what "analysis" is... Breaking a system or process into it's components to study each one independently.

Partly. But crucially, no. You're describing a myopic kind of reductionism. For instance, amplifiers interact with cables which interact with speakers which interact with the amplifier, and the room, which then feeds back into all the components. Studying them independently, or in a different configuration, misses all kinds of important stuff about how they behave when combined. Hence the importance of measuring everything.

But listening to "every system in every room" is no easier.

Point taken!

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #9
]If you measure 10% distortion and I measure 0.01% distortion, one of us is doing something wrong!

Not at all: if I measure 10% in my system and you measure 0.01% in yours, both results are valid - in fact, such field testing often reveals issues that were not present in lab conditions. Feedback.

DVDdoug is correct. Let's say we're bench testing an amplifier: A spec like ".001% THD"  by itself is meaningless. At what frequency were you measuring, at what signal level, and what kind of load was the amplifier driving at the time? Ideally, relevant background information of the sort should be included, and if the test was done correctly, you or I should be able to arrive at very similar figures if we replicate the test conditions. We can also get a pretty good idea of how it'll perform in "real world" conditions by say, measuring it as it drives various reactive loads, not just an 8 ohm resistor.

If audio specs got a bad rap in the past, it's because manufacturers were only too willing to publish impressive-sounding figures out of context, else measured under iffy and undemanding conditions.

As for cables, no mystery there: At audio frequencies, it really does come down to Resistance [R], Capacitance [C] and Inductance [L], and at typical lengths used in a home system, those are all really low, with the runs from phono cartridge to preamp being most critical with respect to C. Nevertheless, it's pretty easy to convince a roomful of people that a change of cables makes a surprisingly big sonic difference, at least until you tighten up your test procedures 

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #10
]If you measure 10% distortion and I measure 0.01% distortion, one of us is doing something wrong!

Not at all: if I measure 10% in my system and you measure 0.01% in yours, both results are valid - in fact, such field testing often reveals issues that were not present in lab conditions. Feedback.

DVDdoug is correct. Let's say we're bench testing an amplifier: A spec like ".001% THD"  by itself is meaningless. At what frequency were you measuring, at what signal level, and what kind of load was the amplifier driving at the time? Ideally, relevant background information of the sort should be included, and if the test was done correctly, you or I should be able to arrive at very similar figures if we replicate the test conditions. We can also get a pretty good idea of how it'll perform in "real world" conditions by say, measuring it as it drives various reactive loads, not just an 8 ohm resistor.

Let's run with your idea . . . imagine a world in which every audio component comes with a substantial book containing a 100% accounting of itself - really, completely described by the manufacturer with full and proper declaration of test procedures and context. In order to be scientifically defensible, it would also have to contain parallel sets of measurements in different systems, under different loads, and in combination with different components.
Who would care?
How many buyers would bother to read it?
How would the manufacturer recoup the time and cost of preparation?
And all manufacturers would have to comply to the same standard, otherwise there would be no basis for comparative evaluation.

I'm not saying it wouldn't be better, but we get the world we deserve. (Windows is a fitting OS for a fundamentally fractured world, not incidentally). Practically the entire music-listening market isn't qualified to understand such detailed description, and so things have deteriorated to the point where a speaker manufacturer can claim 25Hz output and not enough people shout back 'What are you talking about'? But it's hard to argue cogently for any position midway between omniscience and 'just follow your ears'.

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #11
Let's run with your idea . . . imagine a world in which every audio component comes with a substantial book containing a 100% accounting of itself
...
How many buyers would bother to read it?

If you're suggesting that science isn't always as entertaining to read about as trips to Italy, bunnies, and caramel-colorations, you'll get no argument from me. As a group, today's hifi journalists may be lacking in technical know-how, but they sure can write persuasively. But a bit of science can help a person to reach their sonic goals faster and more cheaply, thus freeing up time and $$ for trips to Italy, bunnies, and lots of fleur de sel caramels.
I'm not saying it wouldn't be better, but we get the world we deserve.
...
But it's hard to argue cogently for any position midway between omniscience and 'just follow your ears'.

Not to worry, there's still lots of room for subjectivity: Even if all properly-designed electronics sound pretty much the same, it's okay to pay extra for products that seem particularly well-made, attractive, and pleasant to use. And transducers + room acoustics are still a bit trickier to pin down. And let's not forget that the enjoyment of listening itself is still 100% subjective!

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #12
Not to worry, there's still lots of room for subjectivity...
I was going to make this point too. A receiver, for example, might have all the specs I need and measure up great. The same receiver can still, subjectively, be a complete POS that I would not buy. (E.g. the chassis shows off dust, the speaker terminals are cheap and wobbly, the front panel lights are blinding, it's too bulky for my cabinet, the buttons are counter-intuitive and I don't want to stare at this in my living room for 5-10 years before I get a new one.)

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #13
Not to worry, there's still lots of room for subjectivity...
I was going to make this point too. A receiver, for example, might have all the specs I need and measure up great. The same receiver can still, subjectively, be a complete POS that I would not buy. (E.g. the chassis shows off dust, the speaker terminals are cheap and wobbly, the front panel lights are blinding, it's too bulky for my cabinet, the buttons are counter-intuitive and I don't want to stare at this in my living room for 5-10 years before I get a new one.)


Actually, all that matters these days is the on-screen display and the remote control. I go for weeks not looking at my AVR any harder than it takes to see if it is turned on or off. ;-)

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #14
Actually, all that matters these days is the on-screen display and the remote control. I go for weeks not looking at my AVR any harder than it takes to see if it is turned on or off. ;-)
Quite true! If you could tuck the ugly appliance away, most (if not MORE) settings are accessible via on-screen displays and button-crammed bricks they call remote controls. 

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #15
Actually, all that matters these days is the on-screen display and the remote control. I go for weeks not looking at my AVR any harder than it takes to see if it is turned on or off. ;-)
Quite true! If you could tuck the ugly appliance away, most (if not MORE) settings are accessible via on-screen displays and button-crammed bricks they call remote controls. 


In the case of my Denon, there aren't a ton of buttons on the remote while there is a lot of flat shiny space.

Its Yamaha predecessor was more like what you say and it did far less. 

The Denon front panel is proportionately simple and button-spare.

There are zillions of functions that are only available via menus and the OSD.

The Denon does not  look all that bad, but it is tucked away behind frosted glass.

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #16
There are zillions of functions that are only available via menus and the OSD.

Maybe I'm being old-fashioned, but I still want suitably large front panel controls with clear, contrasty markings in case I lose the remote, none of these microscopic multimode buttons with gray-on-gray markings. And for the fussier stuff, give me the option of a web interface so I'm not typing input labels using the remote!!

Ethan Winer's "AES Damn Lies Workshop" Video

Reply #17
There are zillions of functions that are only available via menus and the OSD.

Maybe I'm being old-fashioned, but I still want suitably large front panel controls with clear, contrasty markings in case I lose the remote, none of these microscopic multimode buttons with gray-on-gray markings. And for the fussier stuff, give me the option of a web interface so I'm not typing input labels using the remote!!



The very few buttons on the front panel allow navigation of the menus.

I have the option of viewing the menus on a 60" HDTV, so clarity, contrast, and size are a solved problem.