Recent comments

Hi Sheldon, Thanks for your response on main street bias. I'll deal with your points in reverse order... Firstly, I agree we need more research into conflict mortality - including multidisciplinary studies reviewing application of epidemiological methods to conflict mortality (since this application is not well-validated). The work on main street bias is one such study, and its authors call for more studies. The ILCS study is probably the closest to the Lancet study in terms of methodology (cluster sampling, etc), and its sample of the Iraqi population was far greater than the Lancet's. So if we're interested in different studies on Iraqi mortality, we should be looking at ILCS. As you mention, ILCS's Jon Pedersen states that the focus on mortality (in the Lancet) compared to the more general focus of ILCS is a difference - although Pedersen doesn't accept this explains the large discrepancy in mortality estimate between Lancet and ILCS for the overlapping period. My point about the similarity (in methodology) between ILCS and Lancet is that if one's support of the Lancet figure rests to a large extent on the "established", "standard" nature of the cluster sampling methodology used, then one can't ignore or dismiss the ILCS study without applying double standards. If you discount the ILCS findings because of the focus of its interview questions (etc), then you should equally be able to discount the Lancet estimate on similar bases (ie subjective, unquantifiable bases). Clearly supporters of the Lancet study should not accept this, if they wish to remain consistent. So, the ILCS should be discounted no more lightly than the Lancet study. And yet the ILCS does seem to be ignored or discounted (or at least it's not discussed in many articles on Iraq mortality - including your own lengthy piece). Now, onto main street bias. You say that <i>"even if there was a 2:1 sampling bias, you'd still be left with an estimate of 300,000+ Iraqi deaths."</i> In that event, I assume you'd be in favour of people being <i>informed</i> that the corrected estimate is 300,000? Sampling bias <b>does matter</b> (even if it affects the estimate by less than your above example). It's also important to remember that small biases introduced in the sampling procedure can lead to very different estimations. You state: <i>"The details of the Lancet results suggest that most people have been killed away from home anyway"</i>. To my knowledge, the Lancet study didn't record details of <b>where</b> people were killed. So this is at most an indirect inference. Furthermore, there is no distinction in the Lancet study between civilians and combatants - the ratio between male and female violent deaths may reflect this (ie predominantly male combatants). Either way, no inference from these ratios (including that females are more likely to stay at home) removes the possibility of bias from sampling close to busy streets. Gilbert Burnham in fact <b>accepts</b> that efforts should be made to avoid such a bias. I'm puzzled by the claim that main street bias is refuted by the fact that a start house could be anywhere on a cross street (and that the next 39 houses would take the interviews around the block to a few other streets). Main street bias (as I understand it) is about <i>network distance</i> (not physical distance per se) from main streets. Cross roads are one link away from a main street, the side streets connected to the cross streets are two links, back alleys connected to these are three links etc. Given the methodology (as published) whole sections of neighbourhoods would be excluded in the sampling process. For example, see: http://www.rhul.ac.uk/economics/Research/conflict-analysis/iraq-mortality/schem4.jpg On the other hand, Gilbert Burnham has elsewhere stated that: <i>"As far as selection of the start houses, in areas where there were residential streets that <b>did not cross</b> the main avenues in the area selected, these were included in the random street selection process, in an effort to reduce the selection bias that more busy streets would have."</i> (My bold) http://tinyurl.com/yltzr8 This is a departure from the published methodology. The question remains: how often did they depart from the methodology, and exactly how did they depart from it? If one is interested in accuracy, surely these questions deserve to be answered. Finally, you ask me for "something better" than the "randomly selected main street" technique. How about a completely random selection process such as that conducted by Jon Pedersen in the ILCS survey of Iraq?

I didn't go into more detail on the topic of "main street bias" because my article is already quite long. However, "main street bias" is unlikely to have significantly skewed the Lancet results for the following reasons: <ul><li>In the previous 2004 Lancet study, the researchers randomly selected locations in Iraq by using a GPS device to begin at a randomly-selected longitude and latitude. For the 2006 study, they felt that this completely random system was impractical because going around the country with an electronic device in their hand was likely to be interpreted as military activity, so they used the "randomly selected main street" technique as a means to <i>approximate</i> random selection of a location. It's not perfectly random, but it's a reasonable attempt to achieve randomness given the practical limitations posed by the level of violence and suspicion in Iraq. Can you suggest something better?</li> <li>The researchers didn't limit their visits to homes which were on the "residential street ... crossing the main street." Rather, they began at a randomly-selected address <i>on</i> that residential street and then visited the 40 homes closest to it, which would have taken them onto homes on other streets. Since the home on the residential street that served as the starting point was randomly-selected, there is no reason to expect that it would have been even close to the main street that the residential street intersected. It could be many blocks away. Calling this "main street bias" is in itself a misnomer. The correct term would be "streets that cross main streets bias."</li> <li>No one has presented any evidence, other than pure speculation, to suggest that people who live on main streets (or on residential streets that cross main streets) have been more likely to die than people who live on streets that do not intersect with main streets.</li> <li>The details of the Lancet results suggest that most people have been killed away from home anyway, which would vitiate any main street bias. For example, the male-to-female ratio shows that many more men than women have been killed: 3.4 male deaths for each female death, and 9.8 violent male deaths for each violent female death. If people were being killed at home, you would expect a higher percentage of female deaths, particularly since women tend to stay at home more than men.</li> <li>Even if we set aside all of the above considerations and assume that "main street bias" exists, I find it hard to imagine that it would have been significant enough to substantially alter the study's outcome. Can you give me any meaningful estimate for how much more likely you think it is for someone to be killed who lives on a street which intersects with a main street in Iraq, as opposed to someone who lives on a street that <i>doesn't</i> happen to intersect with a main street? I find it hard to imagine that the ratio could be so much as 2:1, and even if there was a 2:1 sampling bias, you'd still be left with an estimate of 300,000+ Iraqi deaths.</li> </ul> As for Jon Pedersen, his methodology was <i>not</i> similar to the Lancet study, as he himself notes in the URL you provided. For starters, his survey asked about a wide range of health and living conditions, whereas the Lancet study focused exclusively on mortality. That difference alone is in my opinion more likely to produce a discrepancy in outcomes between the two studies than any skewing due to "main street bias." The bottom line in all of this, though, is that we should be having <i>more</i> research like the Lancet study, from other research teams in addition to the Johns Hopkins group. If you question the methodology of the Lancet study, fine; then let's have additional studies that address your methodological concerns. In science, no single study should be presumed definitive, and the question of how many people are dying in Iraq (and what they're dying from) is important enough to deserve multiple studies.