Science, like almost every other aspect of our society, is in the midst of disruption. In that disruption, the very nature of science may be changing. And that is bringing a number of very pertinent questions up.
Two weeks ago I took Malcolm Gladwell to task for oversimplifying science for the sake of a good story. I offered Duncan Watts as a counter example. One reader, Ted Wright, came to Gladwell’s defence and in the process of doing so, took a shot at the reputation of Watts, saying with tongue firmly in cheek, “people who are academically lauded often leave an Ivy League post, in this case at Columbia, to go be a data scientist at Yahoo.”
Mr. Wright (yes, I have finally found Mr. Wright) implies this a bad thing, a step backwards, or even an academic “selling out.” (Note: Watts is now at Microsoft where he’s a principal researcher)
Since Wright offered his comment, I’ve been thinking about it. Where should science live? Is it a sell out when science happens in private companies? Should it be the sole domain of universities? I’m not so sure.
Watts is a sociologist. His area of study is network structures and system behaviors in complex environments. His past studies tend to involve analyzing large data sets to identify patterns of behavior. There are few companies who could provide larger or more representative data sets than Microsoft.
One such company is Google. And there are many renowned scientists working there. One of them is Peter Norvig, Google’s Director of Research. In a blog post a few years ago where he took issue with Chris Anderson’s Wired article signaling the “End of Theory”, Norvig said:
“(Chris Anderson) correctly noted that the methodology for science is evolving; he cites examples like shotgun sequencing of DNA. Having more data, and more ways to process it, means that we can develop different kinds of theories and models. But that does not mean we throw out the scientific method. It is not “The End of Theory.” It is an important change (or addition) in the methodology and set of tools that are used by science, and perhaps a change in our stereotype of scientific discovery.”
Science as we have known it has always been reductionist in nature. It requires simplification down to a controllable set of variables. It has also relied on a rigorous framework that was most at home in the world of academia. But as Norvig notes, that isn’t necessarily the only viable option now. We live in a world of complexity and the locked down, reductionist approach to science where a certain amount of simplification is required doesn’t really do this world justice. This is particularly true in areas like sociology, which attempts to understand cultural complexity in context. You can’t really do that in a lab.
But perhaps you can do it at Google. Or Microsoft. Or Facebook. These places have reams of data and all the computing power in the world to crunch it. These places precisely meet Norvig’s definition of the evolving methodology of science: “More data, and more ways to process it.”
If that’s the trade-off Duncan Watts decided to make, one can certainly understand it. Scientists follow the path of greatest promise. And when it comes to science that depends on data and processing power, increasing that is best found in places like Microsoft and Google.