Skip to main content

Topic: The McGurk Effect (Read 8575 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Dologan
  • [*][*][*][*]
  • Members (Donating)
The McGurk Effect
I know this is probably not what was in mind with the "psychoacoustic phenomena" this forum is intended for, but this sure as hell has to count as one... (if not, feel free to move this wherever is most appropriate)

The McGurk effect -how our *vision* can dramatically affect what we hear- is an effect first described in 1976, but which was recently talked about on a Scientific American podcast. Check it out at SciAm or YouTube.

At least in my case, it is absolutely freaky how well it works and makes a wonderful example how our mind really has the last word on what we would think is up to our ears. Also makes you wonder how spot on this forum is in supporting ABX as a basis for any auditory claim...
  • Last Edit: 12 May, 2008, 07:54:11 PM by Dologan

The McGurk Effect
Reply #1
I remember a study that was a sighted speaker comparison where only the color of the grilles were changed.  Yellow scored the best by a comfortable statistical margin - go figure.  Placebo is extremely powerful.
Was that a 1 or a 0?

  • Axon
  • [*][*][*][*][*]
  • Members (Donating)
The McGurk Effect
Reply #2
That's actually a lot more subtle than I expected. The "da" sound is really ambiguous when looking at the face, and the "ba" sound is really ambiguous when I'm not. But yeah, the effect works.

  • Polouess
  • [*]
The McGurk Effect
Reply #3
That's actually a lot more subtle than I expected. The "da" sound is really ambiguous when looking at the face, and the "ba" sound is really ambiguous when I'm not. But yeah, the effect works.


Dosn't work at all for me 

  • retro83
  • [*][*]
The McGurk Effect
Reply #4
On a somewhat related note, I was linked to this MP3 from one of the related items at the end of that video.
Listen to it with headphones on.

I've never heard such realistic spatial sound, especially in the vertical plane. Really is an odd sensation listening to it.

  • Dologan
  • [*][*][*][*]
  • Members (Donating)
The McGurk Effect
Reply #5

That's actually a lot more subtle than I expected. The "da" sound is really ambiguous when looking at the face, and the "ba" sound is really ambiguous when I'm not. But yeah, the effect works.


Dosn't work at all for me 

Really? Do you hear the same "ba" regardless of whether you are looking at the guy or not? That's interesting...
My girlfriend says she can't hear the D sound, she gets a mix of "ga" and "ba" sound while looking and "ba" when not, but she does perceive the effect.
In my case, while looking at the guy, I hear something between "Ga" and "Da", but most definitely NOT "Ba", no matter how much I concentrate on hearing the "B" sound. On the other hand, as soon as I look away or close my eyes, the "Ba" sound pops up, crystal clear. If I open and close my eyes during the video, I can even get to hear something like "ba da, da ba, ba da" or whatever, depending on the state of my eyelids. Really spooky.

  • Dologan
  • [*][*][*][*]
  • Members (Donating)
The McGurk Effect
Reply #6
On a somewhat related note, I was linked to this MP3 from one of the related items at the end of that video.
Listen to it with headphones on.

I've never heard such realistic spatial sound, especially in the vertical plane. Really is an odd sensation listening to it.

  Whoa, man, indeed! Even EAX has never had such a realistic spatial effect, although, to be fair, it's probably significantly easier to record it than to emulate it that way.

  • Martel
  • [*][*][*][*][*]
The McGurk Effect
Reply #7

On a somewhat related note, I was linked to this MP3 from one of the related items at the end of that video.
Listen to it with headphones on.

I've never heard such realistic spatial sound, especially in the vertical plane. Really is an odd sensation listening to it.

  Whoa, man, indeed! Even EAX has never had such a realistic spatial effect, although, to be fair, it's probably significantly easier to record it than to emulate it that way.

You would have to render the sound from the 3D model of the environment (but using sound refraction properties of materials instead of textures). Perhaps through some sort of sound raytracing (except bass which is not very ray-like).
Even if you created a reliable algorithm for such rendering, you're not likely to see it in discrete sound cards anytime soon since it would probably require more computational horsepower than you can put together on a chip.
IE4 Rockbox Clip+ AAC@192; HD 668B/HD 518 Xonar DX FB2k FLAC;

  • kdo
  • [*][*][*][*]
  • Members (Donating)
The McGurk Effect
Reply #8
On a somewhat related note, I was linked to this MP3 from one of the related items at the end of that video.
Listen to it with headphones on.

I've never heard such realistic spatial sound, especially in the vertical plane. Really is an odd sensation listening to it.

Is it anything different from binaural recordings?



( the "da da" trick works on me perfectly.  )

  • xmixahlx
  • [*][*][*][*][*]
The McGurk Effect
Reply #9
everyone realizes that this is crafted, right? it uses video editing to mux a different audio track onto a video track...

it is fun tho


later

  • Fandango
  • [*][*][*][*][*]
The McGurk Effect
Reply #10
In my case, while looking at the guy, I hear something between "Ga" and "Da", but most definitely NOT "Ba", no matter how much I concentrate on hearing the "B" sound. On the other hand, as soon as I look away or close my eyes, the "Ba" sound pops up, crystal clear. If I open and close my eyes during the video, I can even get to hear something like "ba da, da ba, ba da" or whatever, depending on the state of my eyelids.
Same here. I think it means our visual sense is very dominant over the auditory sense.

everyone realizes that this is crafted, right? it uses video editing to mux a different audio track onto a video track...
Sure, why not. You can call it an audio-visual illusion. You can (re-)create Illusions artificially so that the phenomena can become noticable by the participant, because many illusions occur in nature without us noticing them. Isolating such an illusion, will make it easier or even possible to notice the paradox in perception.

  • Fandango
  • [*][*][*][*][*]
The McGurk Effect
Reply #11
PS: I just found out about this lady... http://en.wikipedia.org/wiki/Diana_Deutsch
I'd like to have one of her CDs!

  • Dologan
  • [*][*][*][*]
  • Members (Donating)
The McGurk Effect
Reply #12
Same here. I think it means our visual sense is very dominant over the auditory sense.

Well, not necessarily. It can also refer to how much vision intervenes in speech recognition, which is a critical ability humans have had to evolve; apparently to such extent that it is able to override auditory input.

everyone realizes that this is crafted, right? it uses video editing to mux a different audio track onto a video track...

Yeah, that's the whole point of it! To illustrate how in case of conflicting information, what you see can trump, or half-trump (with odd results) what you actually hear.

  • raintheory
  • [*][*]
The McGurk Effect
Reply #13
On a somewhat related note, I was linked to this MP3 from one of the related items at the end of that video.
Listen to it with headphones on.

I've never heard such realistic spatial sound, especially in the vertical plane. Really is an odd sensation listening to it.


I think this can be accomplished by using binaural stereo mics with a flat plate seperating them (simulating a head?), and moving the sound back and forth...    Then using a lowpass filter to simulate the sound of it going behind the head, and a highpass filter for in front of the head...    Would this work?   

I am intrigued by this, I do a fair amount with experimental sound works.  Might have to try to see if I can replicate it using this method.

  • Dologan
  • [*][*][*][*]
  • Members (Donating)
The McGurk Effect
Reply #14
I think this can be accomplished by using binaural stereo mics with a flat plate seperating them (simulating a head?), and moving the sound back and forth...    Then using a lowpass filter to simulate the sound of it going behind the head, and a highpass filter for in front of the head...    Would this work?   

I am intrigued by this, I do a fair amount with experimental sound works.  Might have to try to see if I can replicate it using this method.

I am not sure a flat plate would really be necessary to provide left-right directional localization; proper separation of the microphones could probably do the trick alone, although a head simulation would no doubt help. What I am not sure is how exactly sounds could be made as being in front or behind you. In human hearing, I'd guess the shape of the outer ears plays an important role in it. Why would you say the lowpass/highpass filter could provide that effect? Even trickier is probably the elevation localization, at which humans aren't very good anyway. Outer ear/echo effects, perhaps?

EDIT: Ah, instead of just blindly speculating, we should have just checked the wikipedia first  Dummy head recording
  • Last Edit: 15 May, 2008, 06:05:46 AM by Dologan