The end of the scientific method.

I recently met with a friend who reported a discussion that he’d had with a third party about how the scientific method was dead. I am a scientist and I am often challenged about matters scientific by friends who are not scientists. I don't mind being challenged but I find I spend a lot of time educating people about science. Some people have some very odd ideas. Essentially what frustrates me is that many people accept what they are told or read at face value.

So I did a search on Google and found what might have been the seminal article by Chris Anderson, Editor-in-Chief at Wired, on the hypothesis that the scientific method is dead (Wired - 23 June 2008). I think the story is that there is so much data now out there now that we don’t need to ‘understand’ the real world or its systems to have a sense of what might happen – correlation is enough. Not surprisingly, as a scientist I think that is rubbish.

A look at the comments on the story mainly agree that the idea is rubbish. The first comment on the list when I looked agreed with my immediate thoughts on the matter. The first step in science is about making some hypothesis based on the data you have, intuition, or just by thinking. That’s all that the Wired article talks about – seeing patterns and correlations. Anderson’s article says that’s as much as we’ll need from now on whereas scientists use their hypothesis as a starting point to locate more data to test their idea; to make sure they are right. Anderson appears to be saying, don’t do that, just act.
The next responder supports Anderson’s position, saying that the hypothesis driven approach prevents innovation because it frames the problem from the outset. I suppose that they think that means that data outside the frame is not considered. Doing science properly means that hypotheses be rejected if the data doesn’t support them. Being human many scientists filter out data that doesn’t support their ideas, but that doesn’t last very long and importantly will eventually be exposed. Bad hypotheses don’t survive. More to the point, many scientists like the ones he identifies use intuition to get started. Contrary to the response, Newton would certainly have had an hypothesis, Darwin too. Maybe they hadn’t written them down, but their understanding of the world, their predecessors, and their topic directed that intuition towards a line of research.

Finally, I was listening to the radio and a story about ‘Maths and Finance’ about another Wired Magazine article by Felix Salmon (Wired 23 February 2009) – ‘Recipe for Disaster: The Formula That Killed Wall Street’. The center of this story is that some mathematicians came up with a formula that allowed financiers to estimate risk in a new and powerful way. It all went very wrong late last year because the formula's predictive model was based on an imprecise measure of risk, namely market prices. These prices were assumed to have factored in all of the ‘relevant’ data because, you know, that’s what markets do. Unfortunately, the researchers did this because the time series of historical data available was far shorter than the cycle of the event that they were trying to predict – the simultaneous default of lots of mortgages. It’s like trying to estimate the risk of being hit by the 1 in 100 year flood next year when you only have 10years of data.

So the problem with Anderson’s idea (actually it is an hypothesis) is that Wall Street have already tried this, and now we are all worried about our jobs in a global recession. So don’t believe a word of it – the Scientific Method is not dead.