The murmur of the snarkmatrix…

August § The Common Test / 2016-02-16 21:04:46
Robin § Unforgotten / 2016-01-08 21:19:16
MsFitNZ § Towards A Theory of Secondary Literacy / 2015-11-03 21:23:21
Jon Schultz § Bless the toolmakers / 2015-05-04 18:39:56
Jon Schultz § Bless the toolmakers / 2015-05-04 16:32:50
Matt § A leaky rocketship / 2014-11-05 01:49:12
Greg Linch § A leaky rocketship / 2014-11-04 18:05:52
Robin § A leaky rocketship / 2014-11-04 05:11:02
P. Renaud § A leaky rocketship / 2014-11-04 04:13:09
Jay H § Matching cuts / 2014-10-02 02:41:13

The Intelligence Pyramid
 / 

More thought provocations via Khoi Vinh. In this interview with Science of Shopping author Paco Underhill, he drops this nugget:

I think of knowledge as a pyramid. At the bottom of the pyramid is data; the next layer of the pyramid is information; the next layer of the pyramid is intelligence; and the top of the pyramid is wisdom. I like to tell my clients that we’re in the business of giving them intelligence and wisdom, and if they want to collect data, or if they want to collect information and process it themselves, that’s their business.

Of course, this pyramid is hardly Underhill’s invention, but I like that he specializes. I’d swap “knowledge” with “intelligence,” as I have. Totally an aesthetic thing, I just think “intelligence” is a word more suited to apply to the whole structure. Pure data can be characterized, in the CIA sense, as “intelligence,” while “knowledge” is a trickier fit. I like this explanation of the four concepts.

I’d say journalism suffers from not articulating these concepts as decisively as Underhill does. When asked what we’re “in the business of” giving to folks, most journalists would probably shrug and say, “Journalism.” Which is absolutely not a separate plank on the intelligence pyramid, our overinflated egos notwithstanding. (Some would answer “stories,” which I think is a less-than-artful way of dodging the question.) If you squint your eyes a little bit, you could might imagine journalism’s version of this pyramid as Underhill’s version, split into two halves — the “objective” half (data and information), and the “subjective” half (knowledge and wisdom). Squint a little bit more, and you might even see how these concepts form your average newspaper — data and information being the substance of the reporting and presenting process, and knowledge and wisdom being fodder for news analyses, commentaries and editorials.

But I’ve seen reporters recoil at the notion that the foundation for all their work is gathering data. And while most journalists seem to be content with providing mere “information” for a time, 90% of them seem to harbor secret ambitions to impart “wisdom.” It would be worth saying, I think, that actually gathering data is a noble end in itself, as is providing information. It would also be worth giving more journalists access throughout their careers to the fields of knowledge- and wisdom-dispensing. (I.e. Rather more clear subjectivity added to the “objectivity” soup.)

11 comments

Mine are old gripes, but expressing my gripes in terms of this pyramid idea:

Inherent in the idea of knowledge as a capping stone is that the knowledge rests on the data. If you have expressed “knowledge” without any serious exploration and exposition of data, then what you have is a capping stone with no foundation, or what journalism refers to as “opinion”, “commentary”, or more cynically “analysis”. Obviously it is insufficient to make lateral citation (even in abundance) to other people’s opinions as a foundation for knowledge. This is supporting your ill-formed capping stone by tying it to neighboring capping stones, which may or may not have foundations of their own.

Replacing knowledge with opinion is one thing, but worse is replacing data with opinion, a journalism practice that Jon Stewart frequently skewers. The idea that collecting different opinions and then publishing them side by side amounts to collecting data seems to be pervasive. So many stories go “he said this, but his opponent said the other. The End!”. What kind of knowledge am I supposed to build on this kind of foundation?

Amen to that. In fact, I think that’s worth pulling out and expressing as a fundamental tenet of the Intelligence Pyramid as applied to journalism: opinions != data.

The fact that a person or a number of people hold a particular opinion certainly constitutes useful data, but that data can really only provide us information about the people who hold the opinion, not about whatever the subject of the opinion is. An opinion is a subset of knowledge, therefore a byproduct of information. Often, that information rests on flawed data.

It bears repeating that this whole system actually is a pyramid for a reason. Data is foundational, and without it, none of the other pieces is legitimate.

For example:

Data – 23% of people believe the world is rhomboid. Approximately 83% of that 23% were repeatedly dropped on their heads as infants, and a non-overlapping 27% regularly inhale powerful aerosols.

Information – Data indicates a strong correlation between brain damage and head injuries during infancy, as well as the inhalation of powerful aerosols. Data therefore indicates a strong correlation between head-injury- or aerosol-related brain damage and belief in the Rhomboid World Theory.

Knowledge – Rhomboid World Theorists are highly likely to have suffered brain damage.

Wisdom – Don’t believe everything a Rhomboid World Theorist tells you.

Clearly there are all kinds of statistical gaps and obfuscations here, and we lack a properly complete data set to support our knowledge gleanings, but in principle, we’ve made a reasonable progression from the base of the pyramid to the peak.

Now, a typical news story about this might mention that Al Gore urges not to believe the Rhomboid Worlders, and that Michael Crichton urges us to believe them. Examining the foundation of those opinions might get us a lot of useful info on how people come by one belief or another. But it will do nothing to support or subvert the Rhomboid World Theory, and is therefore of limited practical usefulness to a person trying to figure out what to believe.

But this mostly ignores scenarios where knowing what people think or have said is itself news. Talk isn’t always cheap, and people’s opinion’s aren’t always solely revealing of themselves. This is especially true in politics. Say if Robert Gates not only thinks, but actually says, that the US military isn’t winning in Iraq, this doesn’t just give us his opinion — we can also extrapolate from it to suggest his likelihood of confirmation, possible policy changes, potential clashes with other branches of government, etc.

Likewise, suppose my sources are prominent GOP fundraisers, and they all start quietly saying that Rudy Giuliani doesn’t have a chance to win the Presidency. I might disagree with their reasoning, but I don’t necessarily need to buy or even know all of it to conclude that as a consequence, he might have a tougher time if he were to run for President than he would have if those people weren’t saying/thinking that. And so on with speech/opinion in issues of diplomacy, voter polls, etc. You can say that the press plays the gossip game too much, but you can’t deny that there isn’t some payoff to this kind of information-gathering/reporting.

Also, there’s the problem of the hermeneutic circle — the more data you turn into wisdom (founded or unfounded), the more it affects how you see and interpret new and old data. The theory at least is that people who know more (in the sense of wisdom) can also parse more data, both because they can tell what’s relevant and what isn’t, and because they can provide context (match data points). While wisdom might be faulty without good data, you don’t really know what to do with data without wisdom — or intelligence, in Matt’s sense.

Also, I don’t know where this all fits in on the whole journalist vs. blogger snarkfest, but considering that the most piercing criticisms of the blogosphere so far has been that bloggers typically either 1) depend on MSM data-gathering for their entries or 2) lack the wisdom of veteran journalists in traditional media, it makes sense that it would line up somewhere.

(P.S.: Matt, are you trying to catch up to Robin’s post count by the end of the year?)

Dan says…

In spirit I agree with Peter’s and Matt’s comments. I think that journalism would be more valuable if its practitioners were better rewarded for actively questioning the assumptions of their colleagues and those on their beats. In political coverage, I’d much prefer it if more ink was spilled thinking about the implications of candidates’ positions and on issues debates. But I don’t think the intellectual pyramid, or talk of relying more strongly on ‘data’ is necessarily helpful.

Here’s my biggest concern (and it is related to Tim’s invoking of the hermeneutic circle): data (or objective facts, truth, reality, etc) is theory-laden in most of our day to day experiences. We can’t begin to collect data without beginning with some framework that is, essentially, someone else’s conveyed opinions. There is certainly a reality out there, as is evidenced by the smack of the ground against the skulls of young Rhomboid-World theorists, but any given set of representations of that reality is derived from an interpretive framework (which is a form of wisdom or knowledge, opinion, whatever…)

This is where the turtles start going all the way down.

In Matt’s example, the data he begins with can be broken down, if we choose to. For instance, what does it mean to say that “23% of people believe the world is rhomboid?” How do we know this?

What this data really means is that the journalist writing this story has found a sociologist with a good reputation who has published a study in a reasonable journal reviewed by a couple of her peers, who has determined that based on responses by a random sample of individuals she can make this assertion with a level of confidence considered sufficient by her profession. This is not to undermine the utility of, or the validity of the truth claims of, statistics. I just want to show that that data is still the result of someone else’s work.

It’s just work that is generally accepted to be beyond serious dispute. I don’t want journalists to go wild challenging scientific findings (although a bit of that will never hurt), but I do think that in order to succeed this theory of an intelligence pyramid would need to be much more explicit in defining what sorts of things would get to qualify as data and what would not. Then those definitions would have to be defended.

In the end though, I wonder if there wouldn’t be a more direct way to encourage a spirit of questioning assumptions and using ‘data’ (defined loosely) that comes not just from different sides of the political spectrum, but data derived from a wide variety of methods of collection, analysis, interpretation. Maybe we just need journalists who approach every story by saying: what are the many ways of knowing that can be brought to bear on this issue, and how I can put them all together?

To both of your excellent points:

Tim, I don’t actually believe the hermeneutic circle is problematic or paradoxical. It’s a recognition of one essential fact: we can only ever approach truth asymptotically. We can get infinitely closer to it, but we can never exactly grasp it. We can make points that are “true,” but we can also always make points that are truer than those points, and so on, and so on. The pyramid is constantly being sharpened. When we arrive at something like wisdom, we return to the base of the pyramid, to gather more or better data, to refine that wisdom.

The best application of the structure is the scientific method. A few adjustments — we typically think of the scientific method as beginning at the “knowledge” stage, with a question. In reality, we can only arrive at a question (or hypothesis, if you want to start it off there) after a process of gathering data and synthesizing information, however informally. (“Hey, this apple just plopped on my head. It must have come from that tree I’m sitting under.”) The method recognizes that the pyramid must constantly be refined, its equivalent of “wisdom” being the theory, always in danger of replacement or improvement.

Just as we can always continue to sharpen the top of the pyramid, we can also always improve the base of the pyramid, by gathering more or better data, to address Dan’s objections. I would say the question we should ask about data most often isn’t “Is this data acceptable or not?” It’s “How useful is this data? How much more or better data is available?” You’re right that the base of my Rhomboid World pyramid is most likely a study conducted by some sociologist somewhere. Another journalist may come along and derive information from a larger, more precise study with a more random sample, and we’d expect that the resultant wisdom would be sharper as well.

Re: folks’ expressed opinions being useful data. I totally agree, Tim. But I’ll point you back here:

The fact that a person or a number of people hold a particular opinion certainly constitutes useful data, but that data can really only provide us information about the people who hold the opinion, not about whatever the subject of the opinion is.

That Gates says the US military isn’t winning in Iraq helps us generate plenty of quality information about Robert Gates’ prospects as SecDef, but very little about the state or nature of the conflict in Iraq.

I think bloggers can play a role at every level of the pyramid, from gathering data to generating wisdom. None of this is the exclusive provenance of journalists. We do it in concert with a host of partners — from this thread alone we’ve got sociologists, government officials, bloggers, and television personalities.

(I’ll never match Robin’s post count. He’s a machine these days. I’m just taking the opportunity to read some of my RSS feeds for the first time in a month-and-a-half. 🙂 )

Re: Re: folks’ expressed opinions being useful data. Definitely this can be useful (and good examples, Tim), if the opinion is what you want to know.

I don’t think journalists should try to produce the same kind of knowledge that scientists produce. But in both professions it’s critically a matter of first deciding what knowledge you’re trying to produce, i.e. “what is my question?”, and then getting the data appropriate to that question.

So one gripe I had, about a process like surveying opinions instead of getting facts, boils down to: creating the wrong knowledge simply because the right knowledge would require data that is too hard to get.

My other gripe was aimed at the prevalence of “news” or “commentary” segments where the goal from the start is for pundits to give their opinions: creating the wrong knowledge as a goal in and of itself.

In both cases it seems like there is a certain job waiting out there to be done but no one is bothering to do it. The audience is culpable; when I sit in front of the TV, the part of my brain that asks “is this the information I wanted?” turns off and I become grateful for the knowledge produced for me, even though it isn’t the knowledge I woul dhave asked for given the chance. Luckily, my brain doesn’t do this when reading the news, at least not yet.

(Posting from JFK International, where I am waiting for my delayed flight home. Maybe when I get to Oakland at 3 am I will find a hotspot and make more long winded posts as I wait for BART to start running to SF.)

I think the notion that the questions are just as important as the answers — the knowledge — is an important insight.

My biggest gripe with the news these days is not the substance of stories but the very nature of the mainstream news agenda: homogenous, inbred, impatient with long-term, slow-cooking stories (i.e. the important ones).

So while the conventional model of citizen journalism just has citizens doing The Things That Journalists Do, I’d much rather see them begin to explore and explain territory that seems interesting and appropriate to THEM. My hope is that what emerged would look nothing like the AP/NYT consensus news agenda.

So while I love the concept of the pyramid o’ knowledge, I’m not sure who, uh, draws up the blueprint. Which areas of knowledge are we seeking to fill in, on all levels from data to wisdom?

Remember folks, I supply the quantity; Matt supplies the quality.

fernando paulsen says…

Very interesting discussion. If there was something like commandments of journalism, I would put on the top a phrase from a polish born mathematician and physicist, not a journalist. That is Alfred Korzybski’s “the map is not the territory”. We operate communicating through language, a symbolic code. Data is key to start the movement of the communication process, but no matter the amount of data collected we will never be able to replicate the territory, the actual event or reality. Thus knowledge, intelligence and wisdom become vital to convey the context, associations and projections of what is going on, and more suitable to our language symbolic coding than data. Data changes much faster than intelligence and wisdom. Aristotle would flunk a modern 4th grade science class, but he will leave his adult audiences with their mouth open with his lessons in ethics, philosophy and logic. The pyramid serves to categorized the communication processs, but some concepts are more equal than others. As Arnold Toynbee said, “culture is what’s left when you have forgotten everything”. What you can forget is data. Wisdom is what remains.

Fernando, how is your news pregnancy project going? I’m reading “Postwar” by Tony Judt (post to come — it’s amazing) about Europe from 1945-2005, and even more fascinated than usual with the notion of a future laid out before us, if only we could see it.

fernando paulsen says…

Robin, I just came back from Boston with my wife, where we did the roadshow to go to Harvard next year to work on the news pregnancy mapping. So if everything works fine, the Paulsen family (wife and four kids) will leave Chile next july for a year or two to go to a mid career Masters. The events are there. We have to learn to see them. The regular tools of journalism do not serve this purpose. Your word is enough, I can’t wait for the post of Tony Judt’s book: I will pay a visit to Amazon as soon as I stop posting this. Which is now…

The snarkmatrix awaits you

Below, you can use basic HTML tags and/or Markdown syntax.