A study in medical journal The Lancet, widely quoted as an estimate of civilian deaths in Iraq, has been comprehensively debunked by an American political magazine.
The National Journal has called on reporters to be more sceptical of scientific reports, their authors and backers. The Lancet has refused to comment on the Journal’s claims.
The Lancet study estimated that more than 650,000 Iraqis had died since the invasion in 2003 until June 2006 – more than 10 times the estimate by any other group.
In an investigation published this month, the National Journal identified a series of potential flaws in the Lancet piece: in the design and conduct of the study, the lack of transparency in the data, and the backgrounds of its authors and funders.
The study used cluster sampling, counting the number of dead in house-to-house surveys and then inferring an estimate for the national population.
The National Journal argues that the survey was so small that each recorded violent death translated into 2,000 dead Iraqis overall.
Critics quoted in the National Journal story included Fritz Scheuren, a past president of the American Statistical Association, who said the weakest part of the survey was its reliance on unsupervised Iraqi researchers.
Critics have also questioned the 98 per cent response rate in a war-torn country, the lack of supporting data such as birth dates and ages or evidence that death certificates were seen by researchers.
The investigation also attacked those involved in the study and their motives. Key researcher Riyadh Laftas, a former child-health official under Saddam Hussein, refused to answer questions about his methods, only presenting the study at an off-the-record meeting of experts in Geneva, none of whom commented afterwards. Original survey reports have not been released, apparently due to concerns over security.
The National Journal revealed that the $100,000 study was part-funded by George Soros – a former presidential rival to George Bush – through his Open Society Institute. There were three other donors – one anonymous, one unknown to the authors, and the Samuel Robin Foundation.
The magazine’s critique said that the report’s authors had partisan considerations and that Lancet editor Richard Horton had not seen the surveyors’ original data.
‘[Les] Roberts [an author of the report] was hardly the only American to lose confidence in Bush,’said the National Journal. ‘The question is whether he and his team lost their objectivity as scientists as well.”
National Journal reporter Neil Munro told Press Gazette: ‘[Science] professionals just like business executives are vulnerable to pressure. Journalists should treat them with scepticism.”
The Lancet’s Horton told the Journal that if ‘clear evidence of misconduct’was presented to the journal, it would ask for an official inquiry.