Science isn’t Broken; It’s merely Incomplete

Christie Aschwanden writes at length on the question of whether or not Science is Broken in the FiveThirtyEight blog.

Christie points out a lot of good things – science is hard, the incentives are screwy, and even good science can change outcomes based on a different analysis. Based on these points and lengthy interviews, she declares:

I’ve spent many months asking dozens of scientists this question, and the answer I’ve found is a resounding no.

Christie Aschwanden

There’re a few problems with that statement, though, as well as the reasoning Christie uses through the article. The author isn’t incorrect in saying that Science isn’t broken, but that does not imply that Science is doing just fine. While retractions and new results do show the scientific system is working, the rate at which we see them can and should be improved.

My background, if you read the Skeptical Methodologist, is in software. In software, we have a four pillar system of quality – Testing, Peer Review, Static Analysis and Design. Not everyone seems to recognize this, though.

Often you’ll run across engineers who believe that testing is the only way to quality – that peer review and other methods are either wastes of time or just evidence that someone didn’t test hard enough. Test, test, test – it’s a monistic theory of quality that can lead to myopia.

From a quality perspective, there are things that are just really hard to test for. We’re lucky, though, as these things are easy to spot in peer review, or through static analysis, or through design methods. It’s only when we use all the arrows in our quiver that we get the best results and produce the best software.

Likewise, there’s a bit of a scientific monism running around, and I think when we start talking about this monism – science as the only or even best way to truth – we get in trouble. We become close minded about other sources of truth – and we run into the same problems of trying to test for all defects in software. Other sources of truth are valuable not because they compete with science, but because they complement science.

How we set up our scientific models – an issue pointed to in the article – can’t itself really be answered by ‘traditional’ science. It does, however, have guidelines in philosophy, namely hermeneutics. The idea there is that how we chose to see the problem we’re trying to solve affects what answers we might see, and that’s valuable. Double checking that our hypothesis is coherent and that our conclusions properly draw from our results, these borrow from the philosophical field of logic and are not necessarily native to empirical science.

Checking our peer’s results, not necessarily looking for fraud but just understanding another human being can catch things that all the rigor in the world might miss, that’s the social circle of our lives, not necessarily empirical science. Looking at the higher system of science, the systems of incentives and what reporting body reports to whom, what prevalent theories – and their owners – might have to gain or lose in prestige by newer competing theories – this is all politics. Political and Economic scientists may have good things to say here in making the process more effective, but not traditional empirical science.

Some might argue that good peer review, unbiased modelling, and fixing incentive structures, that’s all part of ‘good’ science, and when you don’t do it, that’s ‘bad’ science. This runs afoul of the ‘no-true-Scotsman’ fallacy, unfortunately, as basically no matter what one detractor might say about science, the proponent will merely say “no, no, no, that’s not TRUE science. TRUE science avoids that.”

Instead, I see science as but one means of discovering truth, and I see the hard sciences in particular as methods using mathematical theories backed with instrumented empirical evidence, gathered by large enough means to run statistical tests on. I define this kind of science narrowly, and by that definition, this kind of science is incomplete. We need other sources of truth to compliment it – testing with our peer review, static analysis with our design methods – to make it better.

Asking scientists whether science is broken misses the point – anyone with a political mind would tell you that. You need to be asking everyone involved in the system, as well as actually properly define what system it is you’re trying to talk about. Science, with a big letter S, needs more input from more fields – it can’t go it alone. We can’t keep just adding on new methods and rigor to what we have and claim that NOW we have GOOD science – we need to acknowledge that our path to truth is always an improving process, and not merely because of new evidence.

New procedures, new incentives, new methods and new models will all be developed, and those all improve what we know about the world too when combined with good empirical science.

When fighting the monster that is our ignorance about the natural world, we need to use every arrow in the quiver.


Did Ayn Rand hit the Glass Ceiling?

While the author and philosopher Ayn Rand may have happily been in favor of the glass ceiling, describing herself as a chauvinist, this doesn’t exactly set her apart amongst other writing colleagues.

Other’s who wrote during the same era included an unrepentant Nazi and a couple that routinely slept with underage teenagers.

There isn’t a philosopher alive or dead that doesn’t have a commentator accusing them of being contradictory, or who’s private lives were weird and not really in line with what we expect. Why is Rand held to some higher standard?

We can compare her as a woman to Simone de Beauvoir, but we have to be careful. De Beauvoir both had a path to fame through her lover, Sartre, as well as was French. More importantly, she seemed to espouse views that were “okay for a woman to have”. Even today you see women more or less lead towards Women’s studies or Feminism in philosophy – as if they didn’t have comments on metaphysics that had nothing to do with estrogen. While we’ve made headway, there still is a sneaking suspicion in many departments that if you’re studying a woman philosopher, it’s going to be about Feminism.

Rand wrote on economics and ethics. She also wrote liberally unpopular views, though not too far afield of Neitzche. Speaking of which, she gets flak for not crediting Neitzche for much of her ‘objectivism’, however, since when do Philosophers ever properly credit each other? Why is it interesting and fun to find Luther’s hidden influences on Heidegger yet Rand is expected to dutifully cite her sources like some school girl we’ve allowed into the boys club?

Don’t get me wrong, I’m not particularly interested in Randian philosophy. But in terms of having an impact, being widely read, the difference between her and Marx is one of degree rather than kind. Still, she’s derided as “not a real philosopher” and a terrible writer – even though history has spoken and she apparently writes in a way entertaining enough for many folks.

I don’t have an answer to this question, but it is worth asking – would she have had as much flak if she were a man? I mean, the chauvinism probably would have dismissed as just being “how people thought back then”. And we’d probably have let “him” off the hook on having some self-contradictions, just as we have most other philosophers. Certainly, there’d be plenty of academic skepticism – “he” would still be writing against the popular worldviews at the time.

Ayn Rand, the woman, though probably gets a double dose of “get the fuck out” in that she wasn’t talking about the things we “allowed” women to talk about back then, and moreover, she didn’t agree with what the men who were allowed to talk about those things thought. So she was a hack, a fad (that still keeps going and going), a false prophet whom we can criticize as truly hysterical.

Indeed, upon reflection perhaps Rand is a great example of how Philosophy, too, is Politics – no more or less than Science. Either she didn’t back the right party, or she wasn’t in the right ‘group’, more likely both.

Science Is Politics

Great coverage of how science works regarding Chronic Fatigue Syndrome.

Science here, and in real life, means two things.

Science is a hermeneutic. It’s a focus on proof-by-induction, empiricism, mathematical formalism and falsifiability.

But it’s also something capable of being analyzed by another hermeneutic: politics. Science is something done by scientists, by human beings. And they’re susceptible to bias.

I find the chronic fatigue syndrome focus particularly fascinating because you can see a parallel to the so-called obesity epidemic: science that, when you really look at it, is relatively meaningless: no study has actually shown long-term success, but each one is twisted and spun to imply long term success is both easy and obtainable, thereby implying that the victim of these diseases is to be blamed.

Of course, there’s that clumsy hermeneutic again – science should never be about blame, it should be about what is. Politics is more often about blame, so we should be warned whenever any study could be used to imply fault.

Let’s not even start on the replicability crisis hitting psychology right now.

Is Dieting Evil?

Social Psychologist Jonathan Haidt uses an interesting framework in his argument that liberals and conservatives fundamentally have different values – or at least, different weights on five universal values.

One of the values, purity, stands out. It’s what a social psychologist might invoke to explain religious prescriptions to avoid pork, the halo effect we get around doctors and other health care professionals, or why we have the adage ‘cleanliness is next to Godliness’. Ultimately, it describes the sense of moral good we do when we do ‘pure’ things – why we naturally assume natural fibers are better than synthetic ones, why we revile instinctively at pollution, why baptism is seen as a ‘cleansing’ of sin.

Keep that in mind, and consider the following – in a small study of 60 participants, it was found that those who bought organic foods tended to display anti-social behavior.

In other words, there is some small bit of evidence that when you use your willpower to do good in one area of life, you give yourself license to be more selfish in other areas of life.

Similar to the above, people’s purchasing and consumption of organic food – here considered a moral good under the label of purity – needed less of at least two other of the moral ‘flavors’ outlined in Haidt’s theory – “reduction of harm” and equality.

But organic food is not the only way we eat to be pure. In fact, I’d argue it’s not even the main way we eat to be pure. Today, during a conversation, this thought struck me as someone said, justifying their purchase of egg whites and vegetables, “I’m trying to be good.”

They were dieting or following a particular diet anyway that required them to use their willpower to deny foods they wanted.

If you gave me five minutes, I could probably name two dozen people I know personally who are attempting to ‘be good’ with what they eat. This rarely materializes in them eating ‘more’ of anything, despite us being told to eat more fruits and vegetables. Instead, it’s always ‘less’ of something. To be good, they need to eat less fat, or sugar, or alcohol, or salt, or dairy, or gluten, or non-organic foods, or caffeine, or beverages other than water. The list goes on and on.

Nevermind the ineffectiveness of most dietary approaches to get any sort of sustainable happiness – what kind of damage is our fascination with diets doing to society as a whole? If let’s say, one-quarter of the whole of American society is currently dieting, how much less ‘good’ is there to go around? Are we giving less to charity and more to low fat, organic food companies in raw dollars than is socially maximizing? Are we ignoring our spouse or our children’s needs because we were ‘good’ today and avoided salt? Are we less challenging of racist and bigoted beliefs that we hold because we ate turkey bacon today?

There is some evidence that this may be the case. This is a philosophy blog, not a scientific one, so I can’t assign p-values here. Instead, speculation is welcome. Does dieting make us, well, more evil? Does being hangry all the time make us less able to see the needs of others?

I, for one, am going to look for this. I’ve noticed more and more that people who treat me in an anti-social manner are almost always somewhat hungry (by their own hand via dieting) or not sleeping. I’ve noticed a lot of back patting when someone lectures me on the fruits of eating organically. They have their moral narrative in life, and they needn’t any longer look for opportunities to act to improve equality, or reduce harm. No, instead, they stick to a particular diet and that diet guarantees they’ll get into Heaven. Or something.