a non-study about transparency

The Pew Research Center, a top-notch think tank, teamed up with Monitor, a premier consulting group, to give us a report on How the Public Perceives Community Information Systems which says, arguably, nothing.

That's not a bad thing. Field studies often end up giving no information (when data doesn't back up our theories). What is a scientific no-no, however, is to conclude findings when you have none - as Pew and Monitor seem to do in this case.

The report's researchers surveyed residents in San Jose, Macon and Philadelphia, asking them a slew of questions about how good a job their local government was doing. After accounting for income and education levels, as well as where residents got their news, one clear picture arose from the findings: people who said their city did a pretty good or very good job of sharing information were also much more likely to be satisfied with their local government and services. (If you like regression tables, you can see the results here.)

It's a clear enough conclusion - but does it tell us anything? If you look closely at all the factors that the researchers accounted for (the "independent variables" in the left side column), you'll notice one suspicious thing - the question about how well a city shares information is the only one that asks residents to make a judgment call about their local government. It's as if you threw in a question that asked "do you think your city government is doing a good job?" (r is efficient, responsive, hard-working or any other positive attribute) and then looked to see how that answer correlated to how satisfied people were with local institutions. You would expect that people who thought highly of their government in one respect would be more likely to look favorably on their government in other respects, no?

That seems to be what this study tells us - that residents who like how their city handles transparency also like how their city handles other services. Not a surprise. Content, satisfied people tend to be content and satisfied all around.

What the study doesn't tell us is if cities that actually do a good job with sharing information also improve residents' faith in their local government. That might be exceedingly difficult - if not impossible - to tease out. (There are a couple of ways to go about it - either comparing dozens of cities and factoring in even more variables, or finding two similar cities when one just happens to introduce a radical transparency policy, without any other governance changes. In both cases, you're still likely to be left with questionable findings.) But at least you'd be trying to find information that tells us something - which this study, unfortunately, does not do.