Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Blind test of live source vs recording/speakers? (Read 20300 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Blind test of live source vs recording/speakers?

Has anything like this ever been attempted?

To clarify, the test would probably look like this:
The blind listener is in a room with both a live source (someone speaking/singing or playing instruments) and some loudspeakers that play recorded material (recorded from the same source). The listener has to identify whether he's listening to the "live sound" or the playback through speakers.

I'm guessing the biggest challenge for transparent speaker playback would be related to directionality and room effects. (No need to be limited to a stereo setup, though.)


Re: Blind test of live source vs recording/speakers?

Reply #2
Has anything like this ever been attempted?

To clarify, the test would probably look like this:
The blind listener is in a room with both a live source (someone speaking/singing or playing instruments) and some loudspeakers that play recorded material (recorded from the same source). The listener has to identify whether he's listening to the "live sound" or the playback through speakers.

I'm guessing the biggest challenge for transparent speaker playback would be related to directionality and room effects. (No need to be limited to a stereo setup, though.)

The biggest challenge is getting the live music to be a close enough duplicate of the recording. I can pretty well guarantee that careful listeners will notice minor changes and timing and intonation.  If the differences are small but still reliably audible, the listener perceives that there is a difference, but is unsure what the difference is. In the real world no performer does exactly the same thing twice.

The diagram that AJ found in the annals of the AES (good job, AJ!)  was IME nothing new when it was published in 1982, People had been talking about this sort of a test for years if not decades before.  It is good example of how to do it right(er) by using the identical same performance. 

The remaining challenge is finding matching anechoic chambers and microphones that are good enough.  Anybody with a lot of spare space and a willingness to write a 7-figure check can play the anechoic chamber game, but I don't know about the microphones.   I've never seen or heard  a microphone that would pass the ABX frequency response matching criteria.

This all serves to show how many light years the performance of electronics, especially good digital is advanced over even the best things acoustical.


Re: Blind test of live source vs recording/speakers?

Reply #3
Not quite the same, but:

http://www.aes.org/e-lib/browse.cfm?elib=11885


Interesting. Do some loudspeakers (and mics) manage to pass the test, aka being indistinguishable?

I can imagine they would, even with that extra mic. There's a mic for the speech anyway (so there's some "coloring" by default) and the listening position is more fixed compared to my live example where the listener could move his head and influence the sound input/direction.


The biggest challenge is getting the live music to be a close enough duplicate of the recording...
Sure, but I didn't have exact duplicates in mind. You'd have various individual performances/recordings and the listener would make a guess every time after hearing one ("live or recorded?").

Re: Blind test of live source vs recording/speakers?

Reply #4
Acoustic Research used to give regular "live vs. recorded" demonstrations all the time back in the 1960's.  They used a string quartet recorded in an open field.  They also used AR speakers (equalized, though).  The quartet would play for awhile and then would pretend to play while the speakers took over.  As I recall it used to fool a lot of people, though it was hardly double blind.

Wharfedale also gave some live vs recorded concerts using an entire orchestra back in the 1950's.

Actually I think it goes back even further if memory serves,  Edison gave live vs recorded events using his original gramophone and the listeners of the day could not tell the difference between the gramophone and the live singer.   It may have been someone else than Edison though, my memory comes from a book I read many many years ago.  Here's a link that seems to confirm my memory but I have no idea if the article has any real authority behind it.  http://www.audiocircle.com/index.php?topic=82842.0
Ed Seedhouse
VA7SDH

Re: Blind test of live source vs recording/speakers?

Reply #5
Hearsay, or possibly even didn't-hearsay, but I think I remember reading of a such a test in phonograph or gramophone days where it was said that the musicians were coached to sound like the recording.
The most important audio cables are the ones in the brain

Re: Blind test of live source vs recording/speakers?

Reply #6
Interesting. Do some loudspeakers (and mics) manage to pass the test, aka being indistinguishable?
Yes. The problems with your type of proposed test, is the recording system won't capture what your 2 ears hear in the same room, or close to what your ears hear, unless it's something like this: http://www.aes.org/e-lib/browse.cfm?elib=9136. So right off the bat, you would be asking the speaker(s) to reproduce what isn't there, for sake of comparison.

cheers,

AJ
Loudspeaker manufacturer

Re: Blind test of live source vs recording/speakers?

Reply #7
http://seanolive.blogspot.nl/2010/07/why-live-versus-recorded-listening.html

After a 1916 live-versus-recorded demonstration in Carnegie Hall, the New York Evening Mail stated “the ear could not tell when it was listening to the phonograph alone, and when to actual voice and reproduction together.

This might be a clue (together)
Anyways it proves that audio journalism in 1916 was already at the same  high standards we enjoy today :)
TheWellTemperedComputer.com


Re: Blind test of live source vs recording/speakers?

Reply #9
Interesting. Do some loudspeakers (and mics) manage to pass the test, aka being indistinguishable?

Probably not. To be acoustically transparent (indistinguishable from perfect)   a device must have better frequency response and lower distortion than even the best speakers and microphones.

The detailed specification of what frequency response, noise and distortion must be for sonic transparency is complex, but it is usually shorthanded as being frequency response +/- 0.1 dB 20 Hz - 20 KHz, less than ca. 0.1 % THD or IM  and noise 80 dB down or better.

Doable with  analog electronics and digital, but  very challenging to say the least, for anything that operates in the acoustical domain.

The biggest challenge is getting the live music to be a close enough duplicate of the recording...

Sure, but I didn't have exact duplicates in mind. You'd have various individual performances/recordings and the listener would make a guess every time after hearing one ("live or recorded?").

I doubt what you could learn very much in that context. Every time you hear a difference, how do you prove that its not due to your sloppy setup?  I say that having actually tried it.  My days aren't long enough for that sort of inherently flawed screwing around.


Re: Blind test of live source vs recording/speakers?

Reply #10
I do recall a Gearslutz conversation in which people were waxing very lyrical about the speakers being really like having the musicians in the room. This might have been studiophiles, but it was professionals, not audiophiles, and earth was regained with a bump when someone asked if anyone in the conversation could ever seriously not know, blind, if they were listening to musicians or their monitors.

But hyperbole can be fun.

The most important audio cables are the ones in the brain

Re: Blind test of live source vs recording/speakers?

Reply #11
You could probably fool me in a concert hall or music hall where the room has a big effect on the sound.    However, I have been to plays where the music was "canned" (usually with live vocals) and I've usually been disappointed.    You could certainly fool me at a stadium rock concert where the virtually all of the live sound comes through the P.A.   

In my living room I think it would be a lot more difficult to foo me.     I wouldn't want a rock band in my living room and a full-orchestra wouldn't fit...  I wouldn't want the sound of a full-orchestra either.  ;)  

Re: Blind test of live source vs recording/speakers?

Reply #12
In a couple of months I intend to do a single blind live vs. recorded session at a hifi show. An internet search brought me to this topic. I've been an occasional lurker for years and I decided it was time to join. I really like that Hydrogenaudio is very science-based.

For the test I'm partnering with an audiophile record label and I have access to an anechoic chamber. We will be using very accurate and quite powerful active monitor speakers. I believe we are well-equipped to make it work!

Of course, this is a marketing stunt. Our aim is to demonstrate the expertise of the recording professionals as well as the quality of the audio system. Therefore, we hope to achieve that the audience will not hear whether they are listening to live musicians or playback from an anechoic recording. But we don't want to use any tricks, such as having the musicians emulate their recorded selves in any way.

Mr. Krueger, let me start by saying thanks for your ABX comparator and I wish your website were still online. I should have saved the audio files on it!
Above you said you had some experience with this sort of inherently flawed screwing around. I understand that live vs. recorded demo's have their limitations, yet I'd still love to make it work. Would you please share your experience and perhaps give some tips?

I would highly appreciate all comments.

Re: Blind test of live source vs recording/speakers?

Reply #13
Quote
yet I'd still love to make it work. Would you please share your experience and perhaps give some tips?
What kind of music, and is there any sound reinforcement?    ...Obviously, it will be easier to fool the listeners if the "live" sound comes from a PA system.    

It would help to have some distance between the performers and the audience.   I'm not sure how much space you'll have at a hi-fi show unless you have access to a theater.    And to make it "proper", you should be in a theater, or music hall, or whatever is appropriate for the type of music being performed.

And of course,  do your own experiments before going to the trade show.

Re: Blind test of live source vs recording/speakers?

Reply #14
Has anything like this ever been attempted?

To clarify, the test would probably look like this:
The blind listener is in a room with both a live source (someone speaking/singing or playing instruments) and some loudspeakers that play recorded material (recorded from the same source). The listener has to identify whether he's listening to the "live sound" or the playback through speakers.

I'm guessing the biggest challenge for transparent speaker playback would be related to directionality and room effects. (No need to be limited to a stereo setup, though.)

The biggest challenge is getting the live music to be a close enough duplicate of the recording. I can pretty well guarantee that careful listeners will notice minor changes and timing and intonation.  If the differences are small but still reliably audible, the listener perceives that there is a difference, but is unsure what the difference is. In the real world no performer does exactly the same thing twice.

The diagram that AJ found in the annals of the AES (good job, AJ!)  was IME nothing new when it was published in 1982, People had been talking about this sort of a test for years if not decades before.  It is good example of how to do it right(er) by using the identical same performance. 

The remaining challenge is finding matching anechoic chambers and microphones that are good enough.  Anybody with a lot of spare space and a willingness to write a 7-figure check can play the anechoic chamber game, but I don't know about the microphones.   I've never seen or heard  a microphone that would pass the ABX frequency response matching criteria.

This all serves to show how many light years the performance of electronics, especially good digital is advanced over even the best things acoustical.



You don't need to worry about performance differences, since the listener could be played a single long piece and switch between it at points. As the piece continued the listener would be able to switch source back and fourth between live and recorded. If the piece was never repeated then you'd be unable to examin what one was recorded.

And just for the record, anechoic chamber hire is around £1500 per day in the U.K.  It's not cheap, but it's not breaking the bank in the name of experimentation.

The microphone would kill you tho, I would assume none are transparent like that. I do have numerous microphones with a flat frequency response, but they tend to sound "bad". I can't quite understand why, but I would guess they are very directional

Re: Blind test of live source vs recording/speakers?

Reply #15
Interesting. Do some loudspeakers (and mics) manage to pass the test, aka being indistinguishable?

Probably not. To be acoustically transparent (indistinguishable from perfect)  a device must have better frequency response and lower distortion than even the best speakers and microphones.

The detailed specification of what frequency response, noise and distortion must be for sonic transparency is complex, but it is usually shorthanded as being frequency response +/- 0.1 dB 20 Hz - 20 KHz, less than ca. 0.1 % THD or IM  and noise 80 dB down or better.

Doable with  analog electronics and digital, but  very challenging to say the least, for anything that operates in the acoustical domain.

Hold on, didn't you previously state that this type of transparency wasn't achievable on analogue equipment?  And also, +/- 0.1dB seems extremely tight for a human to decipher any audible difference. I'd like to see a source for those numbers.

Re: Blind test of live source vs recording/speakers?

Reply #16
In a couple of months I intend to do a single blind live vs. recorded session at a hifi show. An internet search brought me to this topic. I've been an occasional lurker for years and I decided it was time to join. I really like that Hydrogenaudio is very science-based.

For the test I'm partnering with an audiophile record label and I have access to an anechoic chamber. We will be using very accurate and quite powerful active monitor speakers. I believe we are well-equipped to make it work!

Of course, this is a marketing stunt. Our aim is to demonstrate the expertise of the recording professionals as well as the quality of the audio system. Therefore, we hope to achieve that the audience will not hear whether they are listening to live musicians or playback from an anechoic recording. But we don't want to use any tricks, such as having the musicians emulate their recorded selves in any way.

Mr. Krueger, let me start by saying thanks for your ABX comparator and I wish your website were still online. I should have saved the audio files on it!
Above you said you had some experience with this sort of inherently flawed screwing around. I understand that live vs. recorded demo's have their limitations, yet I'd still love to make it work. Would you please share your experience and perhaps give some tips?

I would highly appreciate all comments.

I'm told that this was done way back then with shellac  78s and horn gramaphones, and achieved amazing results. I'm also told that the "live" performers had special training to sound like the record. Yes: a marketing stunt.

Even the phenomenon of stereo itself requires a certain willing suspension of disbelief.
The most important audio cables are the ones in the brain


Re: Blind test of live source vs recording/speakers?

Reply #18
Interesting. Do some loudspeakers (and mics) manage to pass the test, aka being indistinguishable?

Probably not. To be acoustically transparent (indistinguishable from perfect)  a device must have better frequency response and lower distortion than even the best speakers and microphones.

The detailed specification of what frequency response, noise and distortion must be for sonic transparency is complex, but it is usually shorthanded as being frequency response +/- 0.1 dB 20 Hz - 20 KHz, less than ca. 0.1 % THD or IM  and noise 80 dB down or better.

Doable with  analog electronics and digital, but  very challenging to say the least, for anything that operates in the acoustical domain.

Hold on, didn't you previously state that this type of transparency wasn't achievable on analogue equipment?

Please Read carefully for a change. You should distinguish between analog  electronics which is what I said and analog media which appears to be what you are thinking about. To clarify, analog electronics includes things like modern op-amp based SS electronics (amplifiers, crossovers, etc.) that outperforms the specs I mentioned, often by orders of magnitude.

Analog media includes LPs and analog tape, which most definitely do not perform to the standards I mentioned.

Quote
And also, +/- 0.1dB seems extremely tight for a human to decipher any audible difference. I'd like to see a source for those numbers.

Since you have repeatedly refused to honor such requests from me, back at you. :-(

For the polite and sincere lurkers around here, the best information I have on this topic is in this AES paper:

Clark, David L., "High-Resolution Subjective Testing Using a Double-Blind Comparator", Journal of the Audio Engineering Society, Vol. 30 No. 5, May 1982, pp. 330-338.

http://www.aes.org/e-lib/browse.cfm?elib=3839




Re: Blind test of live source vs recording/speakers?

Reply #19

You don't need to worry about performance differences, since the listener could be played a single long piece and switch between it at points. As the piece continued the listener would be able to switch source back and fourth between live and recorded. If the piece was never repeated then you'd be unable to examin what one was recorded.


Obviously, you've never done any serious ABXing for subtle differences.  :-(

The music between any two different points in a performance is usually different music, and is generally pretty easy (on the ABX scale of extreme listener sensitivity to differences) to distinguish from the music between any other two different points.

People who don't have real world experience with hearing subtle differences that are still audible would likely never know this. It is one of those things like hearing +/- 0.1 dB level or frequency response differences that you have to personally do, to believe.

Re: Blind test of live source vs recording/speakers?

Reply #20

Mr. Krueger, let me start by saying thanks for your ABX comparator and I wish your website were still online. I should have saved the audio files on it!

You can still find them on the Wayback machine:

https://archive.org/web/

There were 3 PCABX.COM web sites that were linked together under the covers to take advantage of promotional deals on web sites of a certain size in that day:  www.pcabx.com, www.cdabx.com and www.pcabx-pr.com.



Re: Blind test of live source vs recording/speakers?

Reply #21

You don't need to worry about performance differences, since the listener could be played a single long piece and switch between it at points. As the piece continued the listener would be able to switch source back and fourth between live and recorded. If the piece was never repeated then you'd be unable to examin what one was recorded.


Obviously, you've never done any serious ABXing for subtle differences.  :-(

The music between any two different points in a performance is usually different music, and is generally pretty easy (on the ABX scale of extreme listener sensitivity to differences) to distinguish from the music between any other two different points.

People who don't have real world experience with hearing subtle differences that are still audible would likely never know this. It is one of those things like hearing +/- 0.1 dB level or frequency response differences that you have to personally do, to believe.


Believe it or not, many professional musicians can keep tempo. Even with the use of a metronome if need be.

Unfortunately I don't have access to that paper.  Can you please post an excerpt? In my personal experience, people can only begin hearing differences around +/-1dB. My ego is not tied to having super elite hearing.

Secondly I didn't provide measurements because I was telling you that I found it to be transparent - what good would measurements be?

Lastly, do you need to be so verbose that you quote me twice? :-)

Re: Blind test of live source vs recording/speakers?

Reply #22
we hope to achieve that the audience will not hear whether they are listening to live musicians or playback from an anechoic recording. But we don't want to use any tricks, such as having the musicians emulate their recorded selves in any way.
How many and what type of musicians? How many encode/reproduction channels?
Loudspeaker manufacturer

Re: Blind test of live source vs recording/speakers?

Reply #23

You don't need to worry about performance differences, since the listener could be played a single long piece and switch between it at points. As the piece continued the listener would be able to switch source back and fourth between live and recorded. If the piece was never repeated then you'd be unable to examin what one was recorded.


Obviously, you've never done any serious ABXing for subtle differences.  :-(

The music between any two different points in a performance is usually different music, and is generally pretty easy (on the ABX scale of extreme listener sensitivity to differences) to distinguish from the music between any other two different points.

People who don't have real world experience with hearing subtle differences that are still audible would likely never know this. It is one of those things like hearing +/- 0.1 dB level or frequency response differences that you have to personally do, to believe.


Believe it or not, many professional musicians can keep tempo.

You apparently vastly underestimate how tight ABX comparisons actually are.

Common musical tempos range from 20 to 140 BPM.  The fastest of these is less than 3 beats per second.

With ABX, it is pretty common to reliably detect timing errors in music in the range of 1-10 milliseconds. The coarsest of these is 6,000 BPM.

Get back to me when you can find a metronome and musician that can work consistently that precisely.

Please post relevant recordings. We'll get back to you with ABX logs.


Re: Blind test of live source vs recording/speakers?

Reply #24

You don't need to worry about performance differences, since the listener could be played a single long piece and switch between it at points. As the piece continued the listener would be able to switch source back and fourth between live and recorded. If the piece was never repeated then you'd be unable to examin what one was recorded.


Obviously, you've never done any serious ABXing for subtle differences.  :-(

The music between any two different points in a performance is usually different music, and is generally pretty easy (on the ABX scale of extreme listener sensitivity to differences) to distinguish from the music between any other two different points.

People who don't have real world experience with hearing subtle differences that are still audible would likely never know this. It is one of those things like hearing +/- 0.1 dB level or frequency response differences that you have to personally do, to believe.


Believe it or not, many professional musicians can keep tempo. Even with the use of a metronome if need be.

Unfortunately I don't have access to that paper. 

Sure you do. They are available to the public and many corporate and university libraries have them. But if you bought one of your own, it might cost you a fraction of what analog media costs today. :-(

One rule of audiophilia appears to be: "Spend the big bucks on High End audio, but not a penny for common audio test gear or science-based audio information"

Quote
Can you please post an excerpt?



Quote
In my personal experience, people can only begin hearing differences around +/-1dB.

A natural consequence of lack of experience with serious scientific audio testing.

Quote
My ego is not tied to having super elite hearing.

My ego is tied to making true, relevant statements.

Quote
Secondly I didn't provide measurements because I was telling you that I found it to be transparent - what good would measurements be?

They might convince you of what you apparently don't want to believe.