the McGurk effect

On my other blog, I posted a video of a French street performer. I think it's a funny video in and of itself, but I also put English subtitles on it. You can be the judge of whether this makes it more or less funny than before.

I find the effect of the subtitles strangely robust. I don't speak French, but my wife does, and when she watches the clip she says that, ever since she saw the subtitles, she has to concentrate pretty hard to hear the original French. It is like a variation on the McGurk effect. If you watch someone's lips while they are saying "Da", and you listen to someone saying "Ba" at the same time, you will hear something closer to "Ga". This illusion is called the McGurk effect.

In this case, you aren't necessarily watching the mouth of the speaker, but you are reading words that prime you to hear certain sounds more than others. The way I like to think of it (which may be wrong, but I don't have time to study the latest theories right now): the subtitles start to fire up certain nodes in the language networks, and this activity acts as a selective filter for the incoming auditory stimulation.

Link to demonstration of the McGurk effect.

read more

Technology !

Apologies for lack of posting lately. I'm approaching the lockdown phase of my Master's thesis writing.
Chances are, if you've come here, you've already been to the much more popular Mind Hacks and noticed the piece on "neurasthenia" -- but just in case you haven't, I'm linking to it here. It's a fascinating review of some early neurological literature in which the authors expressed concern over the possible mind-numbing effects of some of the newer technologies of the time, most specifically with regards to effects of the quickening of the pace of life that these technologies afforded. This same sort of worry isn't foreign to our generation. The internet is the latest technology to induce nightmares of a future earth populated with flabby, inert cyborgs whose virtual reality has eclipsed the allure of reality itself. Something like that, I guess.

It reminds me of Heidegger's critique of technology. Some folks use Heidegger as an argument against modern technology (i.e. technological devices and their ill effects upon us). Critically, Heidegger's notion of technology is much broader than just the devices that we commonly call technological. When we say "technology" we usually mean "something that makes practical use of advances in our scientific understanding of the world." Heidegger, on the other hand, is referring to a way of living where things appear to us as resources. And if the thing we see resists being seen as a resource, we see it as an obstacle of sorts and set out to find ways to convert it into a resource.

Take file-sharing as an example. Certain advances have made it possible to own an artist's music without paying for it.* And we jumped at the opportunity, didn't we? How did we get here? We can explain it by the lack of moral urgency facilitated by the internet's gift/curse of anonymity. Really, there may be any number of psychological explanations for it. Choose your theory. However you cash it out, I think it is at least reflective of an underlying refusal to see artists as little more than resources (at least while we're involved in stealing their music). Ripping a copy of my favorite band's latest song may feel like I'm giving them a compliment (I could be spending my precious time downloading someone else's music for free, right?), but if it is a compliment, it is backhanded one.

The great danger that Heidegger sees in this is that while in the past cultures may have had different, evolving ways of seeing and understanding the things we encounter in the world, the technological framework is one in which things are gradually divested of meaning altogether. The difference is that in these other, non-technological ways of seeing the world, we got caught up in a hermeneutic circle with things. We developed certain practices toward things based on what they were (and what they were was revealed to us on the basis of fundamental things like our deities, the capabilities of our bodies, the layout of the land, etc.), and these very practices changed the meaning of the things, which again altered our practices toward them. On the other hand, the entire thrust of the technological way of seeing the world is too erase those aspects of things that force us to adopt certain practices (e.g. we erase the aspect of the musician that would make us want to trade something valuable for her artistic work). We need them to be "flexible-shifty", in the words of a great teacher. It's like the hermeneutic circle between us and things loses momentum and falls flat. We're drawn to see things in a way that puts them totally in our power -- as our resources -- and if you're thinking that this sounds like Nietzsche (i.e. will to power) speaking through Heidegger, I think you're right on point.

I'm not too worried about neurasthenia, but I admit that I sometimes do worry that things become more meaningless the better I get at making things mean what I want them to mean. If that last part doesn't sound like a tautology to you, good. If it does, better.

* To be fair, I'm a fence-sitter on this topic. I've been a working musician before, so I know how important it is to get material in the ears of people, even if it means giving it out for free (or happily letting them steal it).

read more

The case of DF (visual form agnosia)

Thanks to Mind Hacks, I found out that my supervisor, Mel Goodale, was featured on ABC Radio National's "All in the Mind" series. In the interview (which you can listen to by clicking here), he talks about a patient named DF, whose unique brain damage (i.e, selective bilateral lesions in the lateral occipital complex due to an episode of hypoxia) resulted in the disruption of her ability to consciously identify objects on the basis of their shape or orientation. In other words, her "vision for perception" was compromised. The fascinating thing is that her "vision for action" was spared. In other words, she can't consciously "see" the shape of objects, but she can interact with objects on the basis of visual information about their shape.


* A view of DF's brain damage (taken from James et al, 2003).

A number of papers have explored the behavioral consequences of DF's pathology. In one of these papers, Goodale and Humphrey (1998) presented DF with a slot that could be rotated and set at various orientations. For the first task, they gave her a card and asked her to match the orientation of the slot by rotating the card in her hand. As illustrated in the figure below, DF was unable to match the orientation of the slot on the basis of her perception of it. For the second task, they simply asked DF to post the card into the slot. Her performance was virtually indistinguishable from that of a healthy control.


* Results are normalized to upright orientation to show deviation from a successful performance.

In the same paper, Goodale and Humphrey report the results of another task, in which DF was asked to pick up flat, non-symmetrical objects. Healthy controls typically accomplish this task by choosing stable grasp points (i.e. opposing vectors on parts of the object with high curvature) for the thumb and index finger, with the object's center of mass laying roughly between the two points. Despite the fact that DF is unable to distinguish between these objects, she is perfect at picking them up in an appropriate way (see figure below). Compare her performance with that of patient RV, who suffers from optic ataxia (caused by bilateral lesions of the occipitoparietal region). People with optic ataxia have preserved vision for perception, but their vision for action is compromised in some way. Thus, while RV is able to distinguish between the objects on the basis of vision, she cannot use that information to guide her grasping movements in an appropriate way. It is important to note that RV doesn't simply suffer from a motor impairment. With her eyes closed, she can successfully reach out and touch locations on her body or pick up objects at remembered locations in her peri-personal space. Her impairment is one of online control of visually guided movements.


* The lines connect the two opposing grasp points used by DF, RV, and a control subject.

The case of DF, when considered along with the case of RV, highlights a double dissociation between vision for perception and vision for action. While DF can accurately guide her hand to objects whose shapes aren't consciously available to her, RV cannot accurately guide her hand to objects on the basis of their shape, even though the shapes of these objects are consciously available to her. This dissociation can be demonstrated in healthy subjects by taking advantage of the fact that the ventral visual processing stream ("vision for perception") is fooled by certain optical illusions, while the dorsal visual processing stream ("vision for action") seems impervious to them. Right now, the best explanation for this difference is that the ventral stream uses allocentric coding (i.e. it deals with spatial relationships between objects in the visual field), while the dorsal stream uses egocentric coding (i.e. it deals with spatial relationships between the viewer's body and target objects in the visual field).

I think it's safe to say that my first exposure to the story of DF marks the beginning of my fascination with the brain.

Works Cited

James, T.W. (2003). Ventral occipital lesions impair object recognition but not object-directed grasping: an fMRI study. Brain, 126(11), 2463-2475. DOI: 10.1093/brain/awg248

Goodale, M.A., Humphrey, G.K. (1998). The objects of action and perception. Cognition, 67(1-2), 181-207. DOI: doi:10.1016/S0010-0277(98)00017-1

read more

Brain and Mind symposium

Over at Channel N+, they have been gradually posting videos from the Brain and Mind symposium at Columbia University. I finally went and checked out the entire program and was delighted to find so many interesting speakers and topics. Click here to see the lineup.

read more