Freakonomics and Data
There’s a really interesting article in the New Republic, “Freaks and Geeks:”
In 2000, a Harvard professor named Caroline Hoxby discovered that streams had often formed boundaries to nineteenth-century school districts, so that cities with more streams historically had more school districts, even if some districts had later merged. The discovery allowed Hoxby to show that competition between districts improved schools. It also prompted the Harvard students to wrack their brains for more ways in which arbitrary boundaries had placed similar people in different circumstances.
…In retrospect, I have come to see this as the moment I realized economics had a cleverness problem. How was it that these students, who had arrived at the country’s premier economics department intending to solve the world’s most intractable problems–poverty, inequality, unemployment–had ended up facing off in what sometimes felt like an academic parlor game?
It’s a very interesting article on the economics of academic economics, and some of the perverse incentives which exist in the field.
Me, I look forward to the day when we have so much data that we can start looking for arbitrary differences and boundaries. I look forward to the day when security has a cleverness problem. No doubt we’ll end up calling it database pharming.