Forecasting tips for analysts

The best book I read in 2015 was probably Superforecasting. It is about a project created by IARPA (Intelligence Advanced Research Projects Activity) to find phenomenal forecasters. Through a competition they identified a team of superforecasters who predicted extremely complicated events with great success. On closer look the author also found out that these people have some common characteristics.

I don’t want to recreate the wheel and make another review of this fantastic book. In fact I would highly recommend people to read the book. There are also some great podcasts that came out recently featuring the author Philip Tetlock. I would recommend the one by Econtalk and another by Freakonomics.

Unknown to many of us, we are actually continuously forecasting in everyday decisions. For example, deciding to use a certain route to go to work to avoid traffic involves forecasting.When we delay buying something in anticipation that price can come down soon, we are making a forecast. People in my profession pretty much earn their bread and butter by trying to make ‘educated’ guesses about the future. Our predictions fail often as much as we get them right. But as two highly respected seniors in this field told me, the difference between a great investment manager and a not so great one can just be a few % of right calls.


Read More

Studies said that …………….

Everyday newspapers report astounding discoveries by scientists and researchers. Some of this research is done upon absolutely new topics. There are also tests done on existing topics where the new study either agrees or disagrees with existing studies done on the subject matter. The ones that get highlighted are usually done by acclaimed people with a lot of credentials. So for laymen like us it is easier to agree with them rather than disagree. After all, who are we to agree or disagree with these experts? 

Now this raises a couple of very interesting questions. At least they did for me. Firstly, how sound are these studies? Are they unbiased and done through rigorous research? Secondly, how do we react when we see these studies?

I got the answer of the first question when I listened to numerous Econtalk podcasts. For the unaware, Econtalk is a talk show hosted by Professor Russ Roberts of George Mason University. What I realized is that these research findings which frequently find the way to the newspaper headlines are not as sound or rigorous as we may have assumed them to be. The distinguished researchers, many from top notch universities of the world can be quite biased. They frequently manipulate the data (data fitting) to ensure that the outcome matches with what they want to show rather than what the truth is.

The newspapers only mention the summary of the findings. However, we rarely have access to the data set used to come to the conclusion. Even if we had access to all the data, how many of us would have to look at it to check for mistakes. If another unbiased scientist using the same methodology came to a similar conclusion then only we can call the study quite robust.

However, just because some people decided to be less truthful does not mean that there are no good researchers. There are people following rigorous methods to come to conclusions. There are also people who cite the weaknesses in their methodology while submitting the results. Thanks to these people human beings have advanced quite a bit.

Now we come to the other interesting part. How do people react when they learn about the outcome of a new research finding? This is where things get even more interesting. I have seen the same person saying that there a new research on healthcare saying that XYZ food has harmful side effects and the same person on a different occasion trashing a separate research on a different food item . What has happened here? “Confirmation bias” has happened.

How do we actually decide which study to believe and which not to? Like I said, it is all about confirmation bias. We just believe the one that we want to believe and choose to ignore all others. Other times, we also do a one person study by ourselves where the only data set is us. Let me take an example of a study which concluded that exercising makes us healthier. Normally I would just quickly relate to myself and try to remember whether that applied to me. Maybe, I did exercise in the past, but I never lost any body fat. I would then quickly say that the research is rubbish because it never applied to me.

What did I do wrong here? Practically everything. Firstly, there are many variables that can influence ones health. My one person study did not control for all the other variables. Secondly, statistics gives a viewpoint about a large sample. I could very easily be an outlier. But that would not necessarily mean that the original research was wrong. Thirdly, I may not have even done the exercises correctly and cheated on form. There could be many more.

Biases are and will remain an integral part of human life. However, it would do all of us good to try to minimize these biases. The first step is to be aware of them. Only then can we try to minimize them while thinking or analyzing.

Read More

borneowebhosting informasiku