My last lecture before the end of term was very interesting though somewhat depressing. A lecturer David Shipworth from the University of Reading, but he is moving to University College London (UCL) after the new year, showed us what happens when you read the fine print on some of those studies.
Those studies are the fundamental basis on which politicians take their decisions and on which national policies are based. The end result is that the research and data we have are very flawed. Very flawed indeed I felt like I shouldn’t really do any more studies. It’s all based on rubbish.
The fact of the matter is, that we don’t know enough. It is difficult to get accurate data for anything. It is even more difficult to propagate the error on the data in complex studies so we can have an idea of how (un)certain the results are. But I would like to see some of this data with error bars on them. They would make the publishers look ridicolous. No one really wants to open that door because it will unleash hell, and no one likes hell.
Although the primary use of those studies are to give policy makers a reflection of the reality on the ground so that they can create policies that would push people to adjust things the way they ought to be, it is often the case that the data comes after the policy has been made. After all the word policy is derived from the same root as the word politics. Once the policy drives the data and the data drives the policy then we go into a self fulfilling loop whereby there is no real fundemental evidence of the need for the policy nor its effectiveness. It all becomes guesswork.
So what is the answer? Fundemental on the ground measurments area good one, but they have their difficulties. They are expensive, something as simple as measuring the temperature of the average house (or the average temperatures of houses) is expensive. You need a sample that is statistically representative of your country of study and then you need to run a datalogger in every room for a long time. Then you need to compile the data and make some assumptions regarding how you combine all the data to come up with the average temperatures.
But not only are they expensive, it is also difficult to know what to measure. You normally measue what you think you should measure, and that again is guess work. You then use what you measured to come up with a theory that probably fits your ideas in the first place because you ignored the things that you have already decided are not important. But then if you are going to measure everything to attempt to arrive at an unbiased view of the data, then you are going to measure a lot. You probably won’t even know that you are missing a lot of stuff.
I have an idea that might be useful to find out what we should measure, I still want to discuss it with some academics and see what they think. If I get good feedback then I will attempt to implement it, maybe just maybe it will work out. Keep your eyes open for a post during January, and remember don’t believe everything you read in an academic paper and in government reports.
I loved that lecture!