gedymin

Robert Wiblin: That’s quite interesting that people who aren’t super educated maybe don’t trust academic journal findings that much. Maybe they don’t respect the credentials, they don’t respect academia. Then you’ve got people who are quite informed who are inclined to defer to scientific research. Then there’s people who know a great deal about academic research and in fact might be more skeptical than many others. They appreciate that many of these supposed findings wouldn’t stand up to scrutiny.

Spencer G: I think that if you think about, let’s suppose you don’t know very much at all about science. You might be skeptical of it as you say. Then you start learning about science and you’re like studies is the way we should answer questions. Then one day you realise the studies often contradict each other and you say, oh no, what do we do with that? Well I know we’ll do randomised control trials because they are better at determining causality and it’s very hard to answer causal questions without randomised control trials. Then you get accepted by those, but then you start realising that those contradict each other.

You’re like oh, okay I know we need a meta analysis. Then you start, which group together these randomised control trials. Then you start looking to those then you realise well there’s publication bias. A bunch of randomised control trials were never published. You actually see a skewed analysis of the data. Then you say oh we need meta analyses that try to do these corrections for publication bias. Then you start learning that some of the methods for correcting publication bias are actually statistically flawed and actually are known to not produce the right answers even though people use them.

Just, how far down the wormhole do you want to go? It can get very disturbing. You’re like, wow okay it’s hard to know what to trust. But the irony and the sad thing to me is that science is so powerful. The actual tools we’re talking about are incredibly powerful, it’s just about how do you execute them in a way, in the real world, with the weird incentives that exist, so that they actually lead to the right answer?