Social Data Analysis

Source: OpenSF

With the rise of easy consumer-oriented data analysis mashup tools like Socrata Social Data Player,Google Fusion Tables, IBM Many Eyes, Swivel, Umapper, Yahoo Pipes and others (we’d like to hear of more)  non-technical people now have access to powerful analysis and visualization tools.  Democratization of data analysis coupled with our tech community further reinforces the idea that govt should be in the business of distributing raw, machine-readable datasets and not editorializing it. This concept is described by Clay Johnson of Sunlight Labs as govt being a wholesaler and others acting as retailers.  We often lack the budget and resources to accomplish what the public can do in a matter of hours or days. This example from Spatial Key demonstrates that the public is often more capable of making sense of the data than those in the walls of govt.

So why aren’t more govt organizations jumping on board this concept? It’s not for technical reasons or know how. It comes down to  fear; losing control of how others will interpret the data.  The desire to control message and how others perceive you. This is not isolated to govt (think of the tobacco industry) but democratic govts have an obligation to be truthful and transparent, warts and all. To those that say data can be used to arrive at biased/inaccurate conclusions, I would make two points: (1) what prevents govt doing the same; and (2) there are enough eyes in the public to identify and refute inaccuracies (Wikipedia).

With social data analysis just starting to mature, we can expect more and more people to participate. It’s now up to govt to liberate data.

-Jay Nath