The murmur of the snarkmatrix…

August § The Common Test / 2016-02-16 21:04:46
Robin § Unforgotten / 2016-01-08 21:19:16
MsFitNZ § Towards A Theory of Secondary Literacy / 2015-11-03 21:23:21
Jon Schultz § Bless the toolmakers / 2015-05-04 18:39:56
Jon Schultz § Bless the toolmakers / 2015-05-04 16:32:50
Matt § A leaky rocketship / 2014-11-05 01:49:12
Greg Linch § A leaky rocketship / 2014-11-04 18:05:52
Robin § A leaky rocketship / 2014-11-04 05:11:02
P. Renaud § A leaky rocketship / 2014-11-04 04:13:09
Jay H § Matching cuts / 2014-10-02 02:41:13

The New Procrastination
 / 

So I’m going to this conference on Saturday. Looks to be a room full of super-smart academics, lawyers, and technologists. And me.

We’re all to write a short position paper ahead of time, so as to facilitate a running start on the conversation; it’s only a day-long event. The papers were due today, and I got mine in, but without leeway to do what I really wanted to: post it here ahead of time.

That’s the new procrastination: not waiting until the last minute (although I did that too), but specifically waiting until it is no longer reasonable to call on your blog readership for comments and critiques.

Anyway, at the conference, I’m on Panel 1, which aims to

review the wide range of what search engines do and their importance in the information ecosystem.

Furthermore,

industry participants, computer scientists, and analysts will flag major trends in search engine technology and try to predict future developments, with the goal of pointing out those trends that will create new conflicts and new litigation.

I actually had a tough time with this; I didn’t want to just make a bunch of random, breezy predictions about video search or super-cool maps or whatever. So I spent the day on Saturday trying to come up with something that really got me excited.

Position paper after the break. It’s already turned in, but of course I’ll have to talk about it (and other things) on Saturday, so comment! Comment!


Robin Sloan, Panel 1: The Search Space

I

November 28, 2005 / Uncategorized

7 comments

First, let me say, Rob, that this is easily one of the most lucid position papers for an academic conference I’ve seen. This will no doubt make you stick out. I especially like when you explain who you are and what EPIC is. Nice.

I would actually be really enthusiastic about google.gov — especially Google Vote. It would be great if you could easily collect, organize, and customize data (almost definitely using RSS) on political candidates: their biographical information, their voting records, their campaign speeches, the latest wire service news, blogs and op-eds, photos, video, etc.

It almost sounds like too much, but it plays to two of Google’s strengths. First, Google’s search engine is particularly adept at separating the signal from the noise — in other words, it’s able to find what’s relevant in an excess of data.

The second strength sounds like it runs counter to the first, but it really doesn’t — Google is also very good at giving you more than you asked for. This particularly applies to Google’s applications apart from its search engines, e.g., Gmail, Google Maps, etc., both in terms of aggregating and integrating different kinds of data and in giving you information you didn’t search for directly but is still relevant — say, the advertisements in Gmail.

Something else worth mentioning (again) is Amazon’s recommendations systems, or the “similar artists” button on AMG. Those engines are particularly good at suggesting information that you didn’t know you were looking for before you found it — in other words, they actually teach you something.

This would be a great feature for Google.gov. On Google Vote, if you look up one candidate’s page, it makes sense to have a link to his/her opponent. Google FOIA could have an “other people who requested this classified information were also interested in…” And so on.

The point is, the perfect search engine doesn’t just locate what we know we want, like something that can find our keys anywhere in the house. It should also be able to give us information we didn’t even know we wanted, until it suddenly became essential.

Matt says…

I can’t help but be amazed at the value of the data Google would generate with a project like this (thinking specifically of Google Vote here). Right now, it’s very hard to track people’s political participation online, because it’s so diffuse — but put together a central hub, with the power & ease of use Google is famous for, track users’ behavior (anonymously, of course) and pow — the most valuable set of political data around.

Exactly, Matt. This seems like exactly what Google is good at: taking something that’s been done before and doing it way better than you ever imagined it could be.

I mean, for sure you’d pipe in data from opensecrets.org, etc. There’s a lot out there to work with.

Aaron says…

Robin, excellent paper! I love how you write: minimal, to the point, and energetic.

I don’t feel I can give much amazing insight you probably haven’t already had, but I can give you my personal reaction to such a possible technology. As you know, I have political interests and I want to express those interests democratically. However, about the only time I vote are the rare local initiatives that I find out before hand, and the big national elections and conventions. To be honest, I only know the name of representative of my district… wait, no I don’t! I can’t help but imagine that this tool, made in the google-way (that is, simple), would empower me to get a handle on what the hell is going on out there politically. I know a lot of the info is already out there, but as you mentioned in the comments, google would organize it and make it accessible. As it is, I am boggled whenever I attempt to gleen any useful information from any .gov site… shit, imagine if google.gov got involved with the DMV?

Sounds like this fellow presenting in panel 4 might be particularly interested in these ideas. Or this fellow, who posted a similar idea on his blog just a few days ago. (Only he takes it, er, a bit further than you might have.)

If Google were able to parse the info from OpenSecrets more effectively, I

Dan says…

I e-mailed these comments to Robin a couple days ago, but I’ll post them here also for discussion’s sake.

I was most intrigued by the implications of the Google FOIA proposal, in large part because I think it points to one of the most striking changes in search: search is now about bringing the world’s information to the internet, instead of the other way around. That seems huge to me. The big things in search will not be faster search algorithms, but will be big, capital intensive enterprises. The new search will not be ethereal.

Here is what I mean:

I am still not so enamored of Google Vote, although I see some possiblities which I’ll get to in a moment. Google Census, similarly, isn’t that exciting because the US census is already available online in all kinds machine readable formats.

But Google FOIA struck a chord. First, I did a bit of web surfing to find out exactly how FOIA requests work. FOIA, it turns out, covers only documents that are produced by executive agencies. It doesn’t cover Congress or the courts, or any state or local stuff. But states and towns tend to have similar laws. When one submits a request they get back ALL of the requested documents and then can do with them as they please (I think this is true. Also, there are corresponding privacy laws that would surely effect whether people’s names actually appeared on such documents, etc. But that is all out of my depth.)

So, now imagine if Google did an FOIA…for everything!

In other words, Google takes the experience and technology that it is gaining from the book scanning project and directs it towards digitizing all government documents as they become available. In the process of book scanning I’m sure Google will already be chock full of gov’t documents, since most major research libraries are also government repositories.

This, now, is revolutionary. Internet search used to mean coming up with incredible technical ways to get at information that was already online. What google has been doing in the last year or so, is to turn itself into a physical enterprise that goes out and turns paper information into internet information. The book project, but also Google Base to some extent, and Google Local, Google Maps, etc. are all part of that process.

Imagine the reaction fo the federal government to a set of FOIA requests asking for pretty much everything. I’m sure that Google would be refused. But they would do it more smoothly: they would have to figure out a way to get the federal government to buy into the project. This would be an enormously expensive project, especially if Google (or whoever) went out of its way to do this sort of work on the state and local level as well. Google would become a massive NGO, of sorts. Google agents working in government offices and archives patiently scanning away. Google investors essentially paying many of the costs of digitizing the American system of government. To think about this more deeply one ought to compare the annual bureaucratic costs of the Fed gov’t with Google’s market cap—and see if it seems feasible.

But in general, this seems like the heart of the insight in this idea: search is not just about looking for info on the internet anymore. It is about making the world’s info fit onto the internet.

But if Google did all that, suddenly I think GoogleVote would become incredible. If I search for my congressman (Rush Holt, D-NJ) and find out not only about the news coverage of him, his campaign planks, and what his critics say, but also how he has voted and how he has engaged with the Department of Energy (or whatever), then I could start to make really informed decisions.

I really like the FOIA idea posted by Dan. (and sorry Robin, I know my comments are too late for your conference.) I just wanted to add to the census idea, that you might want to look into GIS software. It’s mega-cool, and a lot of prof’s studying ecology at MSU use it already, as well as geographers. Essentially, it is a map with multiple layers: you could map a city scape: not just roads, buildings, but underlying sewers, above-ground electric lines, anything. (Or animal populations over time…)

PS We all desperately need to learn to program, well. As I’m finishing my last app to med school, I still know I have to learn how to program at some point. If only I enjoyed sitting in front of glowing boxes more.

The snarkmatrix awaits you

Below, you can use basic HTML tags and/or Markdown syntax.