Scientist A says they do; scientist B says they don’t. How do you decide?
Three researchers have developed a novel method to consider that question: they analyzed the research output of experts who filed a brief in a U.S. Supreme Court case involving violent video games and teens.
You know why it’s novel? Because no scientist would ever have used it.
Their conclusion? Experts who say violent video games are harmful to teens have published much more evidence supporting their claims than have experts on the other side of the debate.
“We took what I think is a very objective approach: we looked at the individuals on both sides of the debate and determined if they actually have expertise in the subjects in which they call themselves experts,” said Brad Bushman…professor of communication and psychology at Ohio State University.
“The evidence suggests that those who argue violent video games are harmful have a lot more experience and stronger credentials than those who argue otherwise.”
You would be forgiven if you thought this was a study about video game violence. It is not. It is a study about how to validate scientific claims, and this method hopes to be used more generally.
Look carefully at the argument being made and the conclusion. The entire study is a cognitive bias, and that bias is called Appeal To Authority.
In essence, the study is studying which group of people is more “expert” about this question. How do you define expert? Experts have published more extensively on this question. I hope it is obvious that this is so flawed as to be useless. The only people who judge expertise by number of publications is social scientists, and that’s why they aren’t considered scientists by anyone else. There’s an entire academic journal devoted to parapsychology that’s recognized by the American Association for the Advancement of Science in which I could happily publish 50 studies. So?
And note that the system has an inherent bias. Whichever side of the question is currently in vogue will be overrepresented by academics and research; eventually this gets corrected as science progresses, but at any moment the number of experts in the field more closely represents the prevailing wisdom than scientific accuracy. Which isn’t to say it isn’t accurate, just that you are less sure that it’s accurate than that it’s the prevailing wisdom.
Nevertheless, this study allowed the author to assert, “”It provides strong support for the argument that video game violence is indeed harmful.”"
No it doesn’t, it’s simply not a link a scientist could make using this method. No one is making any attempt to interpret the studies themselves– that’s not what they want. The point is to be able to quickly screen a social or controversial question for expertise in an effort to control the dialogue on matters no one wants to/can evaluate on their own. I’ll let you imagine how this “novel method” might be put to excellent use in climate change and etc. Science becomes a matter of faith.
No related posts.
Yes.
I’d like to do a study counting up how many studies on each side of the issue (or any given issue) rely on measuring the phenomenon itself, and how many prefer instead to rely on proxies that are more convenient to measure.
In the instant before I vanished up my own recursive irony-hole, I bet the results would be almost as enlightening as the one you quote.
This also reflects our media binarism: there are two sides, and one is right and the other isnt. Note that all of these studies are broken down into two possible conclusions, while I’m willing to bet much of the research on both sides delivers nuanced interpretation of their data or small scale ideas [ie: there is a link between hyper violent first person shooters and the perception of videos of the Iraq war, a study I just made up, but you get the idea].
Which side is right? The side which can muster the loudest shouting, the most voices. This is the perverse quantitative numerologist perception of democracy, opinion, and truth, and it applies to how media actors, more and more underfunded/unwilling to do actual research, addresses problems. Which side are most of my colleagues on?
Clearly, the question was phrased wrong, and the correct answer is that violence causes video games.
Sometimes I get tired of the pessimistic tone of TLP/PO. Surely stuff isn’t as bleak as everyone here thinks it is. Most people are smart, and basically good. Society will probably be just fine.
And then I read this post. Holy shit. “Hey, let’s just count how much evidence there is. The side with more evidence has better scientists. And better scientists are more correct than worse scientists.” This is apocalyptically terrible reasoning, and it will catch on everywhere, because it’s quasi-objective, but by design requires exactly no knowledge of the actual science.
Now, I’m sure people have been convinced by large quantities of wrong evidence before… but to see it done deliberately, as a method…
And I beheld a pale horse, and the name of him that sat on him was Death…
“If you want a picture of the future, imagine a boot stamping on a human face— forever.” George Orwell
But, on the other hand, you do know that any serious scientist who reads this is going to just laugh hard.
Let’s be clear, the study sought to determine how many signers of the “games cause violence” amicus brief have themselves done research on media violence, and how many signers of the “games do not cause violence” brief have done theier own research on media violence. The conclusion was that; “The results showed that 60 percent of the Gruel brief signers (who believe video game violence is harmful) have published at least one scientific study on aggression or violence in general, compared to only 17 percent of the Millett brief signers.”
So the study showed that the signers of the Gruel brief may be forming opinions based on their own research, while the signers of the other brief are forming opinions based on their readings, skimming, or interpretation of someone else’s study.
This isn’t “science.” There isn’t any arguing in science, it shouldn’t be a matter of persuasion. But when social policy dresses itself up as science to cast an opinino the results are screwy.
“The evidence suggests that those who argue violent video games are harmful have a lot more experience and stronger credentials than those who argue otherwise.”
This might be true, but because serious scientists who would argue such things probably don’t care
Pingback: Interesting Links | Fog of Chaos
So, I read this article and this other one in a row, and it made me wonder: what does this one means about climate change? All we know is that a study (similar to this one) was conducted showing that a vast majority of climate scientists agree on man-made ongoing climate change. Of course, as a scientist, I’m supposed to say “all the article proves is that almost every climate scientist agrees in one thing, it doesn’t prove that thing is right”. Still, I don’t see how would one go about making up one’s mind about any scientific issue, other than becoming a scientist in that area, repeting all the experiments and see what they turned out. There are really obvious cases where you can check the methodology and see it’s full of crap, so if I had to study “which side is right”, I guess I’d have to set some metrics that help me decide which papers are crap and which aren’t, and then weight the number of papers by their crappiness, starting by zero, meaning “this paper is complete crap and I won’t take it into account”.
Of course, as a scientist, I’m not supposed to never make up my mind about anything unless I study it in detail for a long time, but that’s kind of a hard way to live.
Any thoughts on this topic?
Well, that basically is why we have meta-analyses, where the constistency and methodology of several studies are laterally examined. This current study rather reminds me of the populist surge in the elections all over Europe. It’s not the evidence, methodology or coherent basis, it’s volume. Who shouts the most frequent loud gibberish wins.
Here are the briefs:
Millet brief:
http://sblog.s3.amazonaws.com/wp-content/uploads/2010/09/AmicusSS.08-1448.pdf
Gruel brief:
http://www.americanbar.org/content/dam/aba/publishing/preview/publiced_preview_briefs_pdfs_09_10_08_1448_PetitionerAmCuLelandYee_AAP_CAandCAPsychAssn.authcheckdam.pdf
The Gruel brief got its signers by looking up the authors of articles that said that video games lead to aggression and asking them to co-sign. Shockingly, this means that the signers had published research on this topic in prestigious journals.
The Millet brief is more of a critique. It basically is pointing out the weakness of the articles that argued video games lead to violence. This brief got apparently got its co-signers by asking random people like Steven Pinker.
The Millet brief seems pretty persuasive to me. Most of the “aggression” found by the other studies is just rambunctiosness. Kids get wound up playing video games. There is also an excellent “what the fuck are you talking about” graph that shows video game purchases and # of violent offenders versus time:
http://missdeandit.files.wordpress.com/2010/07/graph-of-video-game-sales1.jpg?w=529&h=272
It’s interesting that video game experts like the National Institute for Media and the Family’s own Dr. Douglas Gentile make statements like “I prefer the term pathological computer or video game use rather than computer, Internet, or video game addiction.” … while publishing studies like “Video game addiction among adolescents: Associations with academic performance and aggression.”
… but why would they keep talking if they weren’t right?
Why can’t these sorts of researchers get a real job?
There are a million other questions that 1) have an objective answer and 2) could do some good for the world that they could have spent their lives answering.
Of all the questions you could spend your 70-some years on this Earth answering as a researcher…this one? On the bright side, at least he can pay his mortgage.
The problem is that in the last century science has shifted from being a vocation to a career.
Hey now, some of them also teach Psych 101.
It’s one way to get paid … but there’s probably not much actual research going on when a lobbyist front that calls itself an “institute” asks for “research to support the agenda”.
“Scientist A says they do; scientist B says they don’t. How do you decide?”
Obviously, as any intelligent person would realize, you check to see who has published more papers. (sarcasm)
Pingback: Statistical Magic and its uses | Fog of Chaos
Pingback: advantage of filing taxes online
Pingback: Wooden toys plans
Pingback: ADHD symptom
Pingback: how to make yourself last longer in bed
Pingback: garment daily business reports
Pingback: msn cam girl
Pingback: Toko Online
Pingback: bacterial vaginosis treatment
Pingback: bird feeders
Pingback: facts about brazil
Pingback: daftwho
Pingback: grandsumoo
Pingback: prospecting leads
Pingback: Swimwear
Pingback: mlmlead generation
Pingback: kidsbedsheets
Pingback: flat mate finders
Pingback: my blog
Pingback: my site
Pingback: cheap flats London
Pingback: flatmate
Pingback: ensure plus
Pingback: room buddies
Pingback: averagemanadventures
Pingback: sanibel island resorts
Pingback: top facts about jamaica
Pingback: truly free search
Pingback: makesomuchmoneeey
Pingback: watchmen
Pingback: Yosh
Pingback: this site
Pingback: Speakasia online
Pingback: Prizee
Pingback: ddos protection
Pingback: appetite suppressant
Pingback: Best shampoo for dandruff
Pingback: Subliminal Advertising
Pingback: Partnerzusammenfuehrung
Pingback: discount watches
Pingback: free niche site ideas
Pingback: best uk online casinos
Pingback: dallas dwi attorney
Pingback: Goldbarren Kaufen
Pingback: herbsforinflammation
Pingback: Get rid of dandruff
Pingback: Gumby Robbery
Pingback: mountainroseherbs
Pingback: theculinaryherbs
Pingback: hcg diet drops
Pingback: PLR eBooks
Pingback: Private Label Rights
Pingback: banking unusual
Pingback: Voodoo
Pingback: Dekoration
Pingback: bulk email program
Pingback: dating asian women
Pingback: argentina passport
Pingback: betting systems
Pingback: Neck Support
Pingback: baby video monitor
Pingback: crock pot
Pingback: 365 crockpot
Pingback: Asian Tiger Mosquito
Pingback: Iron Deficiency Symptoms
Pingback: Subliminal