spacer image
spacer image

Welcome! You're looking at an archived Snarkmarket entry. We've got a fresh look—and more new ideas every day—on the front page.

November 28, 2005

<< The First AdWord Ever | Now That's a Beat >>

The New Procrastination

So I’m going to this conference on Saturday. Looks to be a room full of super-smart academics, lawyers, and technologists. And me.

We’re all to write a short position paper ahead of time, so as to facilitate a running start on the conversation; it’s only a day-long event. The papers were due today, and I got mine in, but without leeway to do what I really wanted to: post it here ahead of time.

That’s the new procrastination: not waiting until the last minute (although I did that too), but specifically waiting until it is no longer reasonable to call on your blog readership for comments and critiques.

Anyway, at the conference, I’m on Panel 1, which aims to

review the wide range of what search engines do and their importance in the information ecosystem.


industry participants, computer scientists, and analysts will flag major trends in search engine technology and try to predict future developments, with the goal of pointing out those trends that will create new conflicts and new litigation.

I actually had a tough time with this; I didn’t want to just make a bunch of random, breezy predictions about video search or super-cool maps or whatever. So I spent the day on Saturday trying to come up with something that really got me excited.

Position paper after the break. It’s already turned in, but of course I’ll have to talk about it (and other things) on Saturday, so comment! Comment!

Robin Sloan, Panel 1: The Search Space

Iím one of the two producers of EPIC, an eight-minute Flash movie about the future of media. The movie imagines a future super-Google that stores, integrates, and organizes all of your stuff: email, photos, blog posts, movies, music, purchases, subscriptions, social networks, and more.

And thatís pretty much what you hear when people talk about the future of search: Itís about organizing information that matters to you and making your life work better.

Which is, of course, excellent.

But no one ever seems to talk about making our shared public life work better. There is an entire class of information that matters to all of us — itís a fuzzy distinction, but Iíd say itís information that we use to pick political leaders and make community decisions — and it is, right now, generally inscrutable.

So what if Google (which Iíll take as the token for any large, search-focused internet media company) actually took a crack at organizing our democracy?


It might include:

  • Google Vote — introduced in 2008, naturally. Itís as much an improvement over a traditional newspaper voting guide as Google Maps is over a gas-station atlas. Itís clear, dynamic, and comes with an API that gives developers access to a database of candidates and ballot items everywhere.

  • Google Census. Some enterprising mash-up artists have begun overlaying Census data on Google Maps already, but itís just a start, and if youíve ever explored the Census, you know thereís a lot there to work with there. Displaying all that data in one holistic view — a citizenís dashboard — is a Google-scale problem.

  • Google FOIA. The process gets streamlined, automated, and underwritten; with Googleís help, making a FOIA request is as easy as sending an email. The results get scanned and indexed for all to see. Googleís philosophy: Anything that can be FOIAíd should be FOIAíd.

  • Google Redistricting. Come on. You know they want to do it.

Once Google gets into this particular game, though, familiar questions gather new life.

  • When I type my ZIP code into Google Vote and get a map of my district and a list of candidates in return, is it okay if I also see AdWords from PACs listed down the side of the page? Even AdWords from the candidates themselves? Is that logically equivalent to campaign ads on the eleven-oí-clock news — or is it something very different?

  • When search results can determine elections, not just electronics purchases, how should we think about rankings and fairness? Will the party with the best search-engine optimization always win? What happens when algorithm changes have political ramifications?

Sure, a next-generation search platform that stores all your stuff — a true Google Grid — would be great to have. I donít doubt that itís coming.

But as lawyers, scholars, technologists, policymakers, and makers of eight-minute Flash movies, we should always be asking ourselves: To what end?

Our most pressing problems are not misplaced emails and hard-to-find reference books. They are much bigger than that, and it would be a shame, with all the wizardry at our disposal, to ignore them.

So I think our conversation about the future of search should include questions of what it ought to be — and how it might begin to affect the core, not just the periphery, of law and public life.

Posted November 28, 2005 at 8:12 | Comments (7) | Permasnark
File under: Snarkpolicy, Technosnark


First, let me say, Rob, that this is easily one of the most lucid position papers for an academic conference I've seen. This will no doubt make you stick out. I especially like when you explain who you are and what EPIC is. Nice.

I would actually be really enthusiastic about -- especially Google Vote. It would be great if you could easily collect, organize, and customize data (almost definitely using RSS) on political candidates: their biographical information, their voting records, their campaign speeches, the latest wire service news, blogs and op-eds, photos, video, etc.

It almost sounds like too much, but it plays to two of Google's strengths. First, Google's search engine is particularly adept at separating the signal from the noise -- in other words, it's able to find what's relevant in an excess of data.

The second strength sounds like it runs counter to the first, but it really doesn't -- Google is also very good at giving you more than you asked for. This particularly applies to Google's applications apart from its search engines, e.g., Gmail, Google Maps, etc., both in terms of aggregating and integrating different kinds of data and in giving you information you didn't search for directly but is still relevant -- say, the advertisements in Gmail.

Something else worth mentioning (again) is Amazon's recommendations systems, or the "similar artists" button on AMG. Those engines are particularly good at suggesting information that you didn't know you were looking for before you found it -- in other words, they actually teach you something.

This would be a great feature for On Google Vote, if you look up one candidate's page, it makes sense to have a link to his/her opponent. Google FOIA could have an "other people who requested this classified information were also interested in..." And so on.

The point is, the perfect search engine doesn't just locate what we know we want, like something that can find our keys anywhere in the house. It should also be able to give us information we didn't even know we wanted, until it suddenly became essential.

I can't help but be amazed at the value of the data Google would generate with a project like this (thinking specifically of Google Vote here). Right now, it's very hard to track people's political participation online, because it's so diffuse -- but put together a central hub, with the power & ease of use Google is famous for, track users' behavior (anonymously, of course) and pow -- the most valuable set of political data around.

Posted by: Matt on December 1, 2005 at 12:20 AM

Exactly, Matt. This seems like exactly what Google is good at: taking something that's been done before and doing it way better than you ever imagined it could be.

I mean, for sure you'd pipe in data from, etc. There's a lot out there to work with.

Robin, excellent paper! I love how you write: minimal, to the point, and energetic.

I don't feel I can give much amazing insight you probably haven't already had, but I can give you my personal reaction to such a possible technology. As you know, I have political interests and I want to express those interests democratically. However, about the only time I vote are the rare local initiatives that I find out before hand, and the big national elections and conventions. To be honest, I only know the name of representative of my district... wait, no I don't! I can't help but imagine that this tool, made in the google-way (that is, simple), would empower me to get a handle on what the hell is going on out there politically. I know a lot of the info is already out there, but as you mentioned in the comments, google would organize it and make it accessible. As it is, I am boggled whenever I attempt to gleen any useful information from any .gov site... shit, imagine if got involved with the DMV?

Posted by: Aaron on December 1, 2005 at 10:18 PM

Sounds like this fellow presenting in panel 4 might be particularly interested in these ideas. Or this fellow, who posted a similar idea on his blog just a few days ago. (Only he takes it, er, a bit further than you might have.)

If Google were able to parse the info from OpenSecrets more effectively, I’d be all about it. Although I think any moves by Google in this direction would draw much more scrutiny from privacy watchdogs, who’ve so far kept the rantings to a dull roar.

I e-mailed these comments to Robin a couple days ago, but I'll post them here also for discussion's sake.

I was most intrigued by the implications of the Google FOIA proposal, in large part because I think it points to one of the most striking changes in search: search is now about bringing the world's information to the internet, instead of the other way around. That seems huge to me. The big things in search will not be faster search algorithms, but will be big, capital intensive enterprises. The new search will not be ethereal.

Here is what I mean:
I am still not so enamored of Google Vote, although I see some possiblities which I'll get to in a moment. Google Census, similarly, isn't that exciting because the US census is already available online in all kinds machine readable formats.

But Google FOIA struck a chord. First, I did a bit of web surfing to find out exactly how FOIA requests work. FOIA, it turns out, covers only documents that are produced by executive agencies. It doesn't cover Congress or the courts, or any state or local stuff. But states and towns tend to have similar laws. When one submits a request they get back ALL of the requested documents and then can do with them as they please (I think this is true. Also, there are corresponding privacy laws that would surely effect whether people's names actually appeared on such documents, etc. But that is all out of my depth.)

So, now imagine if Google did an FOIA...for everything!

In other words, Google takes the experience and technology that it is gaining from the book scanning project and directs it towards digitizing all government documents as they become available. In the process of book scanning I'm sure Google will already be chock full of gov't documents, since most major research libraries are also government repositories.

This, now, is revolutionary. Internet search used to mean coming up with incredible technical ways to get at information that was already online. What google has been doing in the last year or so, is to turn itself into a physical enterprise that goes out and turns paper information into internet information. The book project, but also Google Base to some extent, and Google Local, Google Maps, etc. are all part of that process.

Imagine the reaction fo the federal government to a set of FOIA requests asking for pretty much everything. I'm sure that Google would be refused. But they would do it more smoothly: they would have to figure out a way to get the federal government to buy into the project. This would be an enormously expensive project, especially if Google (or whoever) went out of its way to do this sort of work on the state and local level as well. Google would become a massive NGO, of sorts. Google agents working in government offices and archives patiently scanning away. Google investors essentially paying many of the costs of digitizing the American system of government. To think about this more deeply one ought to compare the annual bureaucratic costs of the Fed gov't with Google's market cap---and see if it seems feasible.

But in general, this seems like the heart of the insight in this idea: search is not just about looking for info on the internet anymore. It is about making the world's info fit onto the internet.

But if Google did all that, suddenly I think GoogleVote would become incredible. If I search for my congressman (Rush Holt, D-NJ) and find out not only about the news coverage of him, his campaign planks, and what his critics say, but also how he has voted and how he has engaged with the Department of Energy (or whatever), then I could start to make really informed decisions.

I really like the FOIA idea posted by Dan. (and sorry Robin, I know my comments are too late for your conference.) I just wanted to add to the census idea, that you might want to look into GIS software. It's mega-cool, and a lot of prof's studying ecology at MSU use it already, as well as geographers. Essentially, it is a map with multiple layers: you could map a city scape: not just roads, buildings, but underlying sewers, above-ground electric lines, anything. (Or animal populations over time...)
PS We all desperately need to learn to program, well. As I'm finishing my last app to med school, I still know I have to learn how to program at some point. If only I enjoyed sitting in front of glowing boxes more.

spacer image
spacer image