'Semenya advantage' science is wrong. Why isn't it being corrected?
The journal that published the dodgy findings refuses to retract them. Times Select asks some surely obvious questions
The British Journal of Sports Medicine has refused to retract a research paper it published, that claims females athletes with elevated levels of testosterone have an unfair advantage in five athletic events.
This controversial study forms the basis for the policy that South African athletics champion Caster Semenya will have to lower her testosterone levels if she wants to compete in professional sport.
Three scientists – including UCT School of Management Studies’ Professor Ross Tucker – have called for the paper to be retracted, because their analysis shows it includes wrong data.
They have written a new letter calling again for the retraction saying that “unreliable data ... leads to unreliable results”.
But the journal has now told the three scientists it will not retract the article (which would nullify it). In a letter, journal editor Dr Karim Khan said the journal staff had considered Tucker and his colleagues’ views and admitted some of the data used was incorrect, but said “a retraction would be inappropriate”.
Khan did not answer Times Select questions as to why he will not retract the study, despite being sent two e-mails asking for a response.Semenya, who broke the 18-year-old South African record for the 400m race on Saturday, is not currently on hormone therapy to lower her testosterone levels. The International Association of Athletics Federations ( IAAF) regulation kicks in from November.
The paper, published in 2017, is a basis for the IAAF policy that intersex athletes with high levels of testosterone in five events have to have medical treatment to ensure their testosterone is only about double that of 99% of female athletes.
The original study examined athletes’ performances at the World Athletic Championships in 2011 and 2013 and showed that those female athletes with very high testosterone do much better than athletes with normal levels of testosterone in events like the 400m, 800m and the 1,500m.
But Tucker, along with the director of the Sports Governance Centre at the University of Colorado‚ Dr Roger Pielke jnr‚ and University of Oslo professor in cell biology Erik Boye reanalysed data provided to them by the study’s authors, and found it included duplicate times of athletes and times never recorded at the championships.Lead author of the original study Dr Stéphane Bermon, who works for the IAAF, in a letter to the journal admitted his study had 230 incorrect data points. He did not respond to questions from Times Select. Boye said he was stunned that the journal did not retract the paper.
“I have in my 40-year-long career as a scientist and member of editorial boards never experienced anything close to this. I believe all respectable scientific journals would retract a paper immediately, when both the authors and the journal admit that the data are faulty.
“To be specific: they acknowledge that there are serious problems with the data which might affect the conclusions, yet it is retained as a legitimate, scientific paper. This is unacceptable and demonstrates that BJSM is not a serious, scientific journal abiding to the rules common for the rest of the scientific world.
“The paper that the IAAF is leaning on to introduce the high testosterone regulation does not present solid, scientific evidence ... The IAAF cannot use poor science to introduce such a regulation.”
He also said the doctors writing the study are employed by the IAAF, who is using their study for its own policy, which creates a conflict of interest.
The researchers from IAAF have published a follow-up letter in the same journal, this time using what they claim to be the correct figures. The results are not good for IAAF’s case against Semenya.Bermon told a New York Times journalist the results of the second study were the same as the first, and that intersex athletes with very high testosterone still had high advantages for five athletic events.
But the new paper showed that intersex athletes like Caster Semenya have a far lower advantage over female athletes than originally stated. Tucker, Pielke jnr and Boye compared the reanalysis to the original paper, and found that the size of the advantage for the high testosterone group decreased in eight of the 11 running events.
In Semenya’s 800m event, for example, the reanalysis showed an advantage of a high testosterone athlete shrunk from 2.1% in the original study to 1.6%.
“This could be significant when it comes to the Court of Arbitration proceedings, because the arguments there are likely to revolve around the question of how large any advantage is and whether any advantage is unfair? The fact that the original study used incorrect data that exaggerated the advantage may weaken the IAAF’s position and arguments,” said Tucker.
Boye said their work on the two papers will likely be used by Semenya when she challenges the IAAF’s policy at sport’s highest decision making body, the Court for Arbitration of Sport.