The murmur of the snarkmatrix…

August § The Common Test / 2016-02-16 21:04:46
Robin § Unforgotten / 2016-01-08 21:19:16
MsFitNZ § Towards A Theory of Secondary Literacy / 2015-11-03 21:23:21
Jon Schultz § Bless the toolmakers / 2015-05-04 18:39:56
Jon Schultz § Bless the toolmakers / 2015-05-04 16:32:50
Matt § A leaky rocketship / 2014-11-05 01:49:12
Greg Linch § A leaky rocketship / 2014-11-04 18:05:52
Robin § A leaky rocketship / 2014-11-04 05:11:02
P. Renaud § A leaky rocketship / 2014-11-04 04:13:09
Jay H § Matching cuts / 2014-10-02 02:41:13

Reading and the Panda's Thumb
 / 

In the fall of my freshman year of college, I read an essay by Stephen Jay Gould called “The Panda’s Thumb” (drawn, I think, from a book by the same name) for an Introduction to Philosophy class.* The premise was that evolution was best revealed not in examples of perfect adaptation of a species to its environment, but in biological accidents, cobbled-together solutions. The panda’s “thumb,” for example, isn’t a finely tailored opposable digit like the human’s, but a kind of randomly mutated bone spur at the end of the rest, held together by an overstretched tendon where a ligament should be. Evolution doesn’t produce perfect solutions – whenever possible, it uses what’s there, readapting existing features (or exaggerated versions of them) to fit new uses. To use the terminology of the late anthropologist Claude Lévi-Strauss, evolution for the most part isn’t an engineer, creating the perfect tools to fit the job, but a bricoleur, a kind of everyday handyman, perfectly willing to use a butterknife in place of a screwdriver if the butterknife is what’s on hand.

The neuroscientist Stanislas Dehaene, of the Collège de France, has been getting a lot of buzz for his new book Reading in the Brain: The Science and Evolution of a Human Invention, which that reading and writing and evolved in much the same way, making use of existing parts of the visual cortex and rewiring them. What’s more, Dehaene claims that reading and writing’s dependence on a part of the brain that originally evolved to serve other purposes has actually helped determine how reading has emerged historically, and even the shapes of letters themselves. Writing, in other words, isn’t entirely arbitrary – it’s limited by how far our brains can bend.

The neuroscience of writing also suggests that it’s primarily a visual phenomenon, and only secondarily a linguistic one (in the sense of language = speech). But the part of the visual cortex that handles reading relays visual recognition of letters to the speech and motor and conceptual centers of the brain so quickly and efficiently that it almost doesn’t matter; reading becomes a total mental act, integrating nearly all of our mental capacities with split-second timing.

Here’s a summary offered by Susan Okie in her review of the book in the Washington Post:

“Only a stroke of good fortune allowed us to read,” Dehaene writes near the end of his tour of the reading brain. It was Homo sapiens’s luck that in our primate ancestors, a region of the brain’s paired temporal lobes evolved over a period of 10 million years to specialize in the visual identification of objects. Experiments in monkeys show that, within this area, individual nerve cells are dedicated to respond to a specific visual stimulus: a face, a chair, a vertical line. Research suggests that, in humans, a corresponding area evolved to become what Dehaene calls the “letterbox,” responsible for processing incoming written words. Located in the brain’s left hemisphere near the junction of the temporal and occipital lobes, the letterbox performs identical tasks in readers of all languages and scripts. Like a switchboard, it transmits signals to multiple regions concerned with words’ sound and meaning — for example, to areas that respond to noun categories (people, animals, vegetables), to parts of the motor cortex that respond to action verbs (“kiss,” “kick”), even to cells in the brain’s associative cortex that home in on very specific stimuli. (In one epileptic patient, for example, a nerve cell was found that fired only in response to images or the written name of actress Jennifer Aniston!)

This result astonishes me, since I was pretty sure that the one cell = one concept model of the brain — what Douglas Hofstadter calls “the grandmother neuron” theory — had been completely debunked. Apparently, though, there’s a Jennifer Aniston cell? At least for some of us? It might not be the ONLY cell that lights up – but it doesn’t light up for anything else (and appears, at least in this case, to function at either the image OR the written name, suggesting a degree of cognitive interchangability between the two).

These reading cells work differently for words we immediately recognize – like the name of Jennifer Aniston – and those that we don’t (again suggesting that the brain works by macros and shortcuts whenever it can). Jonah Lehrer explains:

One of the most intriguing findings of this new science of reading is that the literate brain actually has two distinct pathways for reading. One pathway is direct and efficient, and accounts for the vast majority of reading comprehension — we see a group of letters, convert those letters into a word, and then directly grasp the word’s meaning. However, there’s also a second pathway, which we use whenever we encounter a rare and obscure word that isn’t in our mental dictionary. As a result, we’re forced to decipher the sound of the word before we can make a guess about its definition, which requires a second or two of conscious effort.

Lehrer also keys in Dehaene’s conclusions about the evolution of writing systems:

The second major mystery explored by Dehaene is how reading came to exist. It’s a mystery that’s only deepened by the recency of literacy: the first alphabets were invented less than 4,000 years ago, appearing near the Sinai Peninsula. (Egyptian hieroglyphic characters were used to represent a Semitic language.) This means that our brain wasn’t “designed” for reading; we haven’t had time to evolve a purpose-built set of circuits for letters and words. As Deheane eloquently notes, “Our cortex did not specifically evolve for writing. Rather, writing evolved to fit the cortex.”

Deheane goes on to provide a wealth of evidence showing this cultural evolution in action, as written language tweaked itself until it became ubiquitous. In fact, even the shape of letters — their odd graphic design — has been molded by the habits and constraints of our perceptual system. For instance, the neuroscientists Marc Changizi and Shinsuke Shimojo have demonstrated that the vast majority of characters in 115 different writing systems are composed of three distinct strokes, which likely reflect the sensory limitations of cells in the retina. (As Dehaene observes, “The world over, characters appear to have evolved an almost optimal combination that can easily be grasped by a single neuron.”) The moral is that our cultural forms reflect the biological form of the brain; the details of language are largely a biological accident.

“Writing evolved to fit the cortex.” On the one hand, it makes perfect sense that a human invention would be limited by human biology – that the visual forms of writing would be limited by our abilities to recognize patterns in the same way that the sounds of letters are limited by the shape and structure of the human mouth.

On the other, it so often seems that writing is BIGGER than we are, or at least independent – that it stands apart and outside of us, like it really was a gift from an Egyptian god – or that it’s so abstract, so removed in modern script from any kind of mimetic resemblance to the world, that it’s a purely arbitrary system, dictated by the requirements of the hand rather than the eye.

The other cool thing about Dehaene’s research? All that brain imaging and reading research and mapping of connections between different parts of the brain has helped him to figure out a neuroscientific way to begin to 1) define consciousness and 2) explain why consciousness is evolutionarily desirable. (Really.)

What I propose is that “consciousness is global information in the brain” — information which is shared across different brain areas. I am putting it very strongly, as “consciousness is”, because I literally think that’s all there is. What we mean by being conscious of a certain piece of information is that it has reached a level of processing in the brain where it can be shared… The criterion of information sharing relates to the feeling that we have that, whenever a piece of information is conscious, we can do a very broad array of things with it. It is available…

In several experiments, we have contrasted directly what you can do subliminally and what you can only do consciously. Our results suggest that one very important difference is the time duration over which you can hold on to information. If information is subliminal, it enters the system, creates a temporary activation, but quickly dies out. It does so in the space of about one second, a little bit more perhaps depending on the experiments, but it dies out very fast anyway. This finding also provides an answer for people who think that subliminal images can be used in advertising, which is of course a gigantic myth. It’s not that subliminal images don’t have any impact, but their effect, in the very vast majority of experiments, is very short-lived. When you are conscious of information, however, you can hold on to it essentially for as long as you wish,. It is now in your working memory, and is now meta-stable. The claim is that conscious information is reverberating in your brain, and this reverberating state includes a self-stabilizing loop that keeps the information stable over a long duration. Think of repeating a telephone number. If you stop attending to it, you lose it. But as long as you attend to it, you can keep it in mind.

Our model proposes that this is really one of the main functions of consciousness: to provide an internal space where you can perform thought experiments, as it were, in an isolated way, detached from the external world. You can select a stimulus that comes from the outside world, and then lock it into this internal global workspace. You may stop other inputs from getting in, and play with this mental representation in your mind for as long as you wish…

In the course of evolution, sharing information across the brain was probably a major problem, because each area had a specialized goal. I think that a device such as this global workspace was needed in order to circulate information in this flexible manner. It is extremely characteristic of the human mind that whatever result we come up with, in whatever domain, we can use it in other domains. It has a lot to do, of course, with the symbolic ability of the human mind. We can apply our symbols to virtually any domain.

Consciousness, in other words, is like writing for the brain – it fixes information that would otherwise be ephemeral, and allows you to perform more complicated operations with it. (Kind of like how we need a pencil and paper to do complicated arithmetic.)

Play with those analogies for a while. I’m going to start reading Dehaene’s book.

*Digression: This class was taught by a prof my friends and I nicknamed “Skeletor,” an ancient woman who couldn’t project her voice beyond the first few rows of the long rows of 50+ desks that passed for a seminar at Michigan State. On some days, she would wear a wrap-around microphone that inevitably dropped down her neck, becoming completely useless. She was always totally oblivious of this. We used to joke that she should wear a live snake wrapped around her neck instead – it would amplify her speech just as well, but everyone would pay rapt attention. I skipped about half of the classes to this class, netting one of my four 3.5s as an undergrad, all of them in my freshman year. If I hadn’t taken Ethics with the great Herbert Garelick the next semester, I’d probably be a math teacher today.

P.S.: I forgot to link to this great Scientific American interview with Dehaene. Here’s a snip:

COOK: In the book, you describe a part of the brain as the “letterbox.” Can you please explain what you mean by that?

DEHAENE: This is the name I have given to a brain region that systematically responds whenever we read words. It is in the left hemisphere, on the inferior face, and belongs to the visual region that helps us recognize our environment. This particular region specializes in written characters and words. What is fascinating is that it is at the same location in all of us – whether we read Chinese, Hebrew or English, whether we’ve learned with whole-language or phonics methods, a single brain region seems to take on the function of recognizing the visual word.

COOK: But reading is a relatively recent invention, so what was the “letterbox” doing before we had written language?

DEHAENE: An excellent question – we don’t really know. The whole region in which this area is inserted is involved in invariant visual recognition – it helps us recognize objects, faces and scenes, regardless of the particular viewpoint, lighting, and other superficial variations.

We are starting to do brain-imaging experiments in illiterates, and we find that this region, before it responds to words, has a preference for pictures of objects and faces. We are also finding that this region is especially attuned to small features present in the contours of natural shapes, such as the “Y” shape in the branches of trees. My hypothesis is our letters emerged from a recycling of those shapes at the cultural level. The brain didn’t have enough time to evolve “for” reading – so writing systems evolved “for” the brain!

17 comments

Tim Carmody says…

Ha! Dehaene talks about Gould and the panda’s thumb just a few pages into the intro. 🙂 The famous Carmody connection-making organ remains tumescent!

This is so. interesting. Thrilling, actually. Great links & a great write-up, Tim.

A few super-salient bits (hard to pick out, b/c it’s all so tasty):

1. A part of the brain called “the letterbox.” Yes please.

2. I want the concept=neuron association to be true, b/c then it means that as you successfully package up new ideas, images, stories, whatever, and get them out to people, you are colonizing other brains. In a literal spatial sense. Yes please to that, too.

3. The consciousness-as-global-information concept is terrific. Also, -as-long-lived-information. It reminds me of that Long Now illustration of the varying speeds of different domains:

And like — not to get spacey here, but — if there really is any sense of a shared consciousness, a society-level consciousness, it’s in our ability to hold on to ideas and information for long periods of time. An avant garde movement is the scaled-up version of a subliminal impression; a myth or a tradition is the scaled-up version of consciousness. (Kinda weak, I know; still noodling it.)

4. There’s a Borgesian scenario to be written in which an erudite blogger, in his long anticipatory posts about new books, actually reconstitutes (preconstitutes?) those books entirely. So publishers start creating fake book descriptions and dangling them in front of this blogger, so that he’ll “write” them, almost by accident.

Wow, yeah I like culture as the next level above individual writing for even greater long-term holding of ideas and sharing of them over domains. Like when a movement like Impressionism spreads from visual art to music.

This short story idea is great and I know this guy who would probably do a great job writing it too…

Socrates tells Phaedrus that it may be noble to sow in the garden of letters, but “nobler far is the serious pursuit of the dialectician, who, finding a congenial soul, by the help of science sows and plants therein words which are able to help themselves and him who planted them….” Socrates grokked the whole concept:neuron thing way back when!

One of the things I find bracing about Dahaene’s book is the idea of neuronal recycling: that systems of neurons in the human brain get repurposed afresh, and that the unfolding of this process looks something like history (well, that’s my take on it, but it flows from the reading). Compared to both evolutionary psychology (which posits a brain built of modules cobbled together in the Pleistocene, all but useless today) or Maryanne Wolf’s developmental pessimism (who worries what the computer’s doing to upset the balance of the literate brain), Dahaene sounds an optimistic and humane note. “The contours of our cerebral processors,” he writes, “far from being rigid, contain an element of plasticity. Their function may change…. Plasticity allows for a cultural diversity that goes beyond a fixed list of modules. Revolutionary inventions like the Internet or the computer mouse, moreover, demonstrate that we are probably far from having discovered all of our cerebral toolbox’s potential.”

More than anything, this comment (& Tim’s post, which I’m still chomping on) made me want to type “wahoo.” So, you know what, I will just do that.

WAHOO!

Tim Carmody says…

Before you Wahoo! again, check out the interview with Dehaene I link to above in the appended PS. I think I tweeted this a few weeks ago — completely forgot I hadn’t blogged about it here.

“We are also find­ing that this region is espe­cially attuned to small fea­tures present in the con­tours of nat­ural shapes, such as the ‘Y’ shape in the branches of trees. My hypoth­e­sis is our let­ters emerged from a recy­cling of those shapes at the cul­tural level.”

!!!!!

If you liked that stuff about the brain’s ability to recycle neurons, you’ll probably LOVE this WIRED article from August 2001 http://www.wired.com/wired/archive/9.08/assist_pr.html

When you think disability, free yourself from the sob-story crap, all the oversize shrieking about people praying for miracles and walking again, or triumphing against the odds. Instead, think puppets. At a basic level, physical disability is really just a form of puppetry. If you’ve ever marveled at how someone can bring a smudged sock puppet to life or talked back to Elmo and Grover, then intellectually you’re nearly there. Puppetry is the original brain-machine interface. It entertains because it shows you how this interface can be ported to different platforms.

If puppetry is the clever mapping of human characteristics onto a nonhuman object, then disability is the same mapping onto a still-human object. Making the body work regardless of physical deficit is not a challenge I would wish on anyone, but getting good at being disabled is like discovering an alternative platform. It’s closer to puppetry than anything else I can think of. I should know: I’ve been at it for 25 years. I have lots of moving parts. Two of them are not my legs. When you think John Hockenberry, think wheelchair. Think alternative platform. Think puppet.

What, none of you remember the Halle Berry neuron?

Tim Carmody says…

Halle Berry fires many, many neurons for me.

The letter A is the head of an ox upturned, horns now pointing down—like in the Buddhist parable, the ox got lost in the world of sensible things (in which the garden of letters is to be found).

I’ve had that Changizi paper Dehaene cites (about natural shapes & written characters) on my desk for two years! It’s deep in the DNA of Urge of the Letter.

Overall good, interesting stuff. The workspace model of consciousness is a nice idea, and one can definitely run with the idea of writing as a way of extending that workspace even farther; there are so many things you can’t properly think through without a pen and paper (or a keyboard) nowadays.

Not such a big deal, but I have to take issue with:

For instance, the neu­ro­sci­en­tists Marc Changizi and Shin­suke Shi­mojo have demon­strated that the vast major­ity of char­ac­ters in 115 dif­fer­ent writ­ing sys­tems are com­posed of three dis­tinct strokes, which likely reflect the sen­sory lim­i­ta­tions of cells in the retina.

This is apparently Lehrer interpreting Deheane interpretating the original papers. Unfortunately it’s a rather gross mis -interpretation/-representation. There is no evidence in these papers that the number of strokes used in written characters is constrained by the visual system. The authors do make the wild (but interesting) conjecture that this reflects constraints in the wiring of visual/association cortex, but they don’t say anything about the retina, and as a retinal scientist I would say the idea that the retina constrains characters to 3 strokes is highly implausible (in so many words: wrong). I wonder whether Deheane/Lehrer’s mistaken reference to the retina is due to not understanding that the word “retinotopic” refers to cortical areas.

Overall I don’t find this line of work on the “evolution” of writing systems very compelling. The authors waffle a bit about what they mean by “evolution”, although they certainly co-opt a lot of the classic vocabulary like “selection pressures”. These papers provide no data on any profound inner workings of the eye/brain. The data go no farther than “writing systems: probably optimized for readability and writability, maybe more for readability”. If these authors really wanted to make a point about “selection pressures” on the development of writing, they ought to have looked at changes in writing systems over time, but they haven’t as far as I can see. What little I know about the evolution of Chinese characters suggests that the opposite is true: more change has occurred over time to optimize writing ease than reading ease. Furthermore, I suspect that if you ran their analysis on ancient versus modern Chinese characters, you would find no increase in optimization for shapes common in the visual world.

Perhaps the authors’ arguments are supposed to apply to earlier periods in the development of writing, but at least for Chinese/Egyptian characters their argument then becomes a ridiculous tautology: “very early characters have a lot of features in common with objects in the natural world” (my version of what this argument would look like, not a real quote from them). Okay guys, but they’re pictograms… (Obviously there are more subtleties here, like the huge visual difference between natural objects and line drawings of natural objects, but the authors don’t seem to have given this much attention.)

Finally, as an aside, I’m going to follow Kenneth Miller and mention that the ultimate “Panda’s Thumb” for human evolution is probably Human Chromosome 2.

Tim Carmody says…

Hmm — if writing has generally evolved from ease of reading towards ease of writing, wouldn’t that make sense as a narrative of the education (I won’t say “evolution” of the letterbox part of the visual cortex)? Initially, writing needs to be highly readable, and specifically needs to bear a close resemblance to those iconic natural shapes that trigger our recognition mechanisms.

However, over time, as literacy education increases and cultures become more used to the recognition of these characters (with maybe — maybe — some evolutionary/socialrealpolitik advantage given to those whose “letterbox” portions of the cortex prove especially adaptable), the characters become more abstract, driven more by ease/simplicity of script than mimetic readability?

There’s also a social evolution here: when writing is the specialized province of scribes (with reading, even in a limited sense, perhaps slightly more common), then script can afford to be highly complex. The pressures of teaching more and more people who are not necessarily specialists – Phoenician merchants, let’s say – creates a pressure (not evolutionary in a biological sense, but socially, sure) to simplify characters, if only to make them easier to teach.

Just a thought.

No, I agree; as a historical/cultural/social story it’s certainly interesting. But there’s not much biology/neuroscience there. And what is there is just intriguing conjecture; there’s no evidence. The impact of visual/motor neurobiology on this development process seems to be overplayed. The use of evolutionary vocabulary is also confusing/misleading.

Right – because the real story is “all this adaptation happened WITHOUT honest-to-goodness genetic evolution! And that’s why it’s so amazing!”

The “global workspace theory of consciousness” sounds so web 2.0…. so cloud computing… I wonder if our understanding of our own minds will only evolve as fast as our own vocabulary for metaphor.

Also, great post.

Reading about how the brain reads is always one of those weird recursive experiences that makes me feel vaguely drunk or like I’m in an elevator with mirrors on all sides.

The part about there being a limited number of strokes for making letters made me think about this video of Claude Closky’s 100-letter Alphabet. The 26 we know plus 74 new letters that in their way look weirdly familiar.

The snarkmatrix awaits you

Below, you can use basic HTML tags and/or Markdown syntax.