Ensuring Health Statistics in Conflict are Evidence-Based: a Response
andrew mack, Human Security Report Project, Simon Fraser University
1 June 2010
We were surprised and disappointed by the tone as well as the content of the editorial by Dr Les Roberts entitled “Ensuring Health Statistics in Conflict are Evidence-Based.” Dr Roberts dismisses the Human Security Report Project’s (HSRP) “Shrinking Costs of War” report as a “poorly done” exercise that received “limited credence in the press and even less in the academic community” and is “the latest and worst” of a series of research interventions by non-public health professionals on global health-related issues.
No evidence was provided for these and other claims that unwarrantably impugn the scholarly reputation of the team that produced the report. The editorial, which even insinuates that the HSRP seeks to make war “more acceptable,” did not even provide a reference to “Shrinking Costs” (which is freely available online at www.humansecurityreport.info) so that readers could judge it for themselves.
Turning to issues of substance, the central claim of the editorial is that the core finding of our “Shrinking Costs of War” report––namely that, "nationwide mortality rates actually fall during most wars"--is invalid. But this rejection, which relies solely on unsubstantiated assertions, is demonstrably untrue.
This finding derived from a review of child mortality rates during periods of warfare in 18 sub-Saharan African countries from 1970 to 2007. The review found that in 78% of armed conflicts in the region the mortality rate was lower at the end of periods of warfare than at the beginning.
The mortality data were drawn from the Inter-Agency Group for Child Mortality (IGME) dataset. The armed conflict data that were used to define the conflict periods were drawn from the International Peace Research Institute Oslo (PRIO) and Uppsala University’s Conflict Data Program (UCDP) battle death datasets. Only countries that had suffered at least 1000 battle deaths and continued to experience 25 or more additional deaths each year were included.
The review drew on under-five mortality data because it is widely accepted that in poor countries where most wars take place, these data are more reliable––and more widely available––than adult mortality data. However, a World Bank study published in 2008 and cited in “Shrinking Costs” found that median adult mortality rates also fell worldwide during periods of conflict.
The claim that mortality rates declined in most war appears highly counterintuitive and––unsurprisingly––met with some skepticism when “Shrinking Costs” was released in January. We were asked how we could be sure that the results for Africa were representative of trends elsewhere in the world. While we were confident of our findings, it was a fair question. To answer it we expanded our review to include all countries around the world between 1970 and 2008.
This time we included only conflicts in the PRIO/UCDP high-intensity, or “war”, category––i.e., those in which there were 1,000 or more battle deaths per year. Altogether 188 countries experienced 480 years of warfare thus defined during this period. We then determined how many of these countries experienced increases in child mortality during the years they were experiencing warfare. The results were even more surprising than the Africa findings.
Worldwide, under-five mortality rates increased in just 5 percent of all of the years in which countries were afflicted by warfare. This counterintuitive finding comes about because recent wars rarely generate enough fatalities to reverse the long-term decline in child mortality that has become the norm throughout the developing world All estimates of child—and adult—mortality are subject to uncertainty, but the data demonstrate unequivocally that national under-five death tolls decline during the overwhelming majority of wars.
The editorial claims that our report, “examines short minor conflicts rather than examining the conflict-affected populations.” This is misleading; it suggests that we chose to examine only minor conflicts. This was not the case. In both the Africa and the global reviews of child mortality in wartime, all conflicts in which the PRIO/UCDP fatality thresholds were exceeded were included––including all the deadliest and longest duration wars. Most conflicts today are indeed minor, but this is because there has been a huge decrease in the deadliness of warfare. It is because today’s wars are less deadly that they have so little impact on nationwide mortality rates.
To demonstrate the degree to which wars have become less deadly we drew on recent research from PRIO/UCDP. Their fatality data reveal that, while the average conflict generated some 10,000 fatalities a year from war-related injuries in the 1950s, in the new millennium the average annual toll had shrunk dramatically to around 1,000. These estimates too are subject to uncertainty, particularly with respect to individual years in which countries experience conflict. But again there is no doubt about the trend.
As for the editorial’s assertion that we should have focused on "conflict-affected populations," this too misses the point. The purpose of our report, as we made very clear, was to examine the changing impact of warfare on national mortality rates, not on rates in war zones. Mortality rates in the latter are of course often many times higher than in non-war areas in the same country––as we also make very clear in the report.
“Shrinking Costs” argues that wartime mortality––including deaths from war-exacerbated disease and malnutrition––has declined dramatically as a result of three factors:
• The change in the nature of warfare over recent decades. • The impact of peacetime health interventions, particularly immunization, on wartime mortality. • The trebling of humanitarian assistance per displaced person since the end of the Cold War.
Nothing in Dr Roberts’ editorial casts any doubt on these conclusions.
References 1. Les Roberts, “Commentary: Ensuring health statistics in conflict are evidence-based,” Conflict and Health 4, no. 10 (2010), 1. 2. Les Roberts, “Commentary: Ensuring health statistics in conflict are evidence-based,” Conflict and Health 4, no. 10 (2010), 1. 3. Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 18. 4. As noted the mortality data were drawn from the consensus estimates of the Inter-Agency Group for Child Mortality (IGME). The IGME data used in the version of the “Shrinking Costs of War” released in January 2010 has subsequently been updated. The new IGME data are used in the global review of child mortality rates in wartime noted below. The IGME data can be viewed here: www.childmortality.org. See also: www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 18. 5. The methodology used by both institutions is essentially the same. It is sometimes referred to as “incident reporting” to distinguish it from survey-based estimation methodologies. Estimates are drawn from a wide range of sources: scholarly studies of particular conflicts, reports by governments and international agencies, humanitarian and human rights NGOs, truth and reconciliation commissions and commissions of enquiry, the media and sometimes retrospective mortality surveys. Researchers then draw up “best estimates” of annual battle death tolls for each country experiencing conflict. This is same process as that typically used by war historians seeking to determine war tolls. For more details see: Bethany Lacina, “Monitoring Trends in Global Combat: A New Dataset of Battle Deaths: Codebook, Centre for the Study of Civil War, International Peace Research Institute, Oslo, October 2005, www.prio.no/sptrans/1732144616/Battle-Deaths%20Codebook.pdf (accessed May 24, 2010). Like all mortality estimation methodologies, including retrospective mortality surveys, this is a process replete with challenges. For a discussion of the problems that confront both incident-reporting methodologies, such as those used by PRIO and UCDP, and population health surveys, such as those used by Dr Roberts, see: Michael Spagat et al., "Estimating War Deaths: An Arena of Contestation", Journal of Conflict Resolution 53, no. 6 (2009): 934-950. While individual conflict tolls can be––and are––contested, the overall decline in the deadliness of conflicts since the 1950s is not in doubt. 6. See chapter 2, footnote 4 in: Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010). 7. Emmanuela Gakidou, Margaret Hogan, Alan D. Lopez, “Adult Mortality: Time for a Reappraisal,” International Journal of Epidemiology 33, no. 4 (2004): 712. 8. Siyan Chen, Norman V. Loayza, and Marta Reynal-Querol, “The Aftermath of Civil War,” World Bank Economic Review 22, no. 1 (2008): 63-85. 9. See: Human Security Report Project, “HSRP Response to: Les Roberts’ Appendices A through C,” 14 April 2010, http://www.hsrgroup.org/images/stories/Documents/HSRP_Response_To_Dr_Roberts_Appendices.pdf (accessed 25 May 2010), 1. A more detailed discussion of our findings will be included in the published version of this report which will be released by Oxford University Press in the fall. 10. PRIO/UCDP have three categories of conflict: “Minor”, “Intermediate” and “War.” The review of African countries used the “intermediate” category; the global review used the “war” category. 11. For this review we used a more recent version of the IGME dataset that had been released after we had completed “Shrinking Costs.” 12. See: Human Security Report Project, “HSRP Response to: Les Roberts’ Appendices A through C,” 14 April 2010, http://www.hsrgroup.org/images/stories/Documents/HSRP_Response_To_Dr_Roberts_Appendices.pdf (accessed 25 May 2010), 1. A more detailed discussion of our findings will be presented in the upcoming published version of this report. 13. Les Roberts, “Commentary: Ensuring health statistics in conflict are evidence-based,” Conflict and Health 4, no. 10 (2010): 1. 14. Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 24. It is of course true that fatality estimates for many wars are subject to considerable uncertainty. But we know of no serious challenge to the central finding of the PRIO battle death dataset, namely that there has been a steep, uneven, but remarkable decline in the deadliness of warfare since 1950. 15. Les Roberts, “Commentary: Ensuring health statistics in conflict are evidence-based,” Conflict and Health 4, no. 10 (2010): 1. 16. See Chapter 2 of: Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010). 17. Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 18-20. 18. How the nature of warfare has changed and why this has reduced wartime fatalities is explained in Chapter 2. See: Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010). 19. Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 2-4.
Conflict and Health
Ensuring Health Statistics in Conflict are Evidence-Based: a Response
1 June 2010
We were surprised and disappointed by the tone as well as the content of the editorial by Dr Les Roberts entitled “Ensuring Health Statistics in Conflict are Evidence-Based.” Dr Roberts dismisses the Human Security Report Project’s (HSRP) “Shrinking Costs of War” report as a “poorly done” exercise that received “limited credence in the press and even less in the academic community” and is “the latest and worst” of a series of research interventions by non-public health professionals on global health-related issues.
No evidence was provided for these and other claims that unwarrantably impugn the scholarly reputation of the team that produced the report. The editorial, which even insinuates that the HSRP seeks to make war “more acceptable,” did not even provide a reference to “Shrinking Costs” (which is freely available online at www.humansecurityreport.info) so that readers could judge it for themselves.
Turning to issues of substance, the central claim of the editorial is that the core finding of our “Shrinking Costs of War” report––namely that, "nationwide mortality rates actually fall during most wars"--is invalid. But this rejection, which relies solely on unsubstantiated assertions, is demonstrably untrue.
This finding derived from a review of child mortality rates during periods of warfare in 18 sub-Saharan African countries from 1970 to 2007. The review found that in 78% of armed conflicts in the region the mortality rate was lower at the end of periods of warfare than at the beginning.
The mortality data were drawn from the Inter-Agency Group for Child Mortality (IGME) dataset. The armed conflict data that were used to define the conflict periods were drawn from the International Peace Research Institute Oslo (PRIO) and Uppsala University’s Conflict Data Program (UCDP) battle death datasets. Only countries that had suffered at least 1000 battle deaths and continued to experience 25 or more additional deaths each year were included.
The review drew on under-five mortality data because it is widely accepted that in poor countries where most wars take place, these data are more reliable––and more widely available––than adult mortality data. However, a World Bank study published in 2008 and cited in “Shrinking Costs” found that median adult mortality rates also fell worldwide during periods of conflict.
The claim that mortality rates declined in most war appears highly counterintuitive and––unsurprisingly––met with some skepticism when “Shrinking Costs” was released in January. We were asked how we could be sure that the results for Africa were representative of trends elsewhere in the world. While we were confident of our findings, it was a fair question. To answer it we expanded our review to include all countries around the world between 1970 and 2008.
This time we included only conflicts in the PRIO/UCDP high-intensity, or “war”, category––i.e., those in which there were 1,000 or more battle deaths per year. Altogether 188 countries experienced 480 years of warfare thus defined during this period.
We then determined how many of these countries experienced increases in child mortality during the years they were experiencing warfare. The results were even more surprising than the Africa findings.
Worldwide, under-five mortality rates increased in just 5 percent of all of the years in which countries were afflicted by warfare. This counterintuitive finding comes about because recent wars rarely generate enough fatalities to reverse the long-term decline in child mortality that has become the norm throughout the developing world
All estimates of child—and adult—mortality are subject to uncertainty, but the data demonstrate unequivocally that national under-five death tolls decline during the overwhelming majority of wars.
The editorial claims that our report, “examines short minor conflicts rather than examining the conflict-affected populations.” This is misleading; it suggests that we chose to examine only minor conflicts. This was not the case. In both the Africa and the global reviews of child mortality in wartime, all conflicts in which the PRIO/UCDP fatality thresholds were exceeded were included––including all the deadliest and longest duration wars. Most conflicts today are indeed minor, but this is because there has been a huge decrease in the deadliness of warfare. It is because today’s wars are less deadly that they have so little impact on nationwide mortality rates.
To demonstrate the degree to which wars have become less deadly we drew on recent research from PRIO/UCDP. Their fatality data reveal that, while the average conflict generated some 10,000 fatalities a year from war-related injuries in the 1950s, in the new millennium the average annual toll had shrunk dramatically to around 1,000. These estimates too are subject to uncertainty, particularly with respect to individual years in which countries experience conflict. But again there is no doubt about the trend.
As for the editorial’s assertion that we should have focused on "conflict-affected populations," this too misses the point. The purpose of our report, as we made very clear, was to examine the changing impact of warfare on national mortality rates, not on rates in war zones. Mortality rates in the latter are of course often many times higher than in non-war areas in the same country––as we also make very clear in the report.
“Shrinking Costs” argues that wartime mortality––including deaths from war-exacerbated disease and malnutrition––has declined dramatically as a result of three factors:
• The change in the nature of warfare over recent decades.
• The impact of peacetime health interventions, particularly immunization, on wartime mortality.
• The trebling of humanitarian assistance per displaced person since the end of the Cold War.
Nothing in Dr Roberts’ editorial casts any doubt on these conclusions.
References
1. Les Roberts, “Commentary: Ensuring health statistics in conflict are evidence-based,” Conflict and Health 4, no. 10 (2010), 1.
2. Les Roberts, “Commentary: Ensuring health statistics in conflict are evidence-based,” Conflict and Health 4, no. 10 (2010), 1.
3. Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010,
http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 18.
4. As noted the mortality data were drawn from the consensus estimates of the Inter-Agency Group for Child Mortality (IGME). The IGME data used in the version of the “Shrinking Costs of War” released in January 2010 has subsequently been updated. The new IGME data are used in the global review of child mortality rates in wartime noted below. The IGME data can be viewed here: www.childmortality.org. See also: www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 18.
5. The methodology used by both institutions is essentially the same. It is sometimes referred to as “incident reporting” to distinguish it from survey-based estimation methodologies. Estimates are drawn from a wide range of sources: scholarly studies of particular conflicts, reports by governments and international agencies, humanitarian and human rights NGOs, truth and reconciliation commissions and commissions of enquiry, the media and sometimes retrospective mortality surveys. Researchers then draw up “best estimates” of annual battle death tolls for each country experiencing conflict. This is same process as that typically used by war historians seeking to determine war tolls. For more details see: Bethany Lacina, “Monitoring Trends in Global Combat: A New Dataset of Battle Deaths: Codebook, Centre for the Study of Civil War, International Peace Research Institute, Oslo, October 2005, www.prio.no/sptrans/1732144616/Battle-Deaths%20Codebook.pdf (accessed May 24, 2010). Like all mortality estimation methodologies, including retrospective mortality surveys, this is a process replete with challenges. For a discussion of the problems that confront both incident-reporting methodologies, such as those used by PRIO and UCDP, and population health surveys, such as those used by Dr Roberts, see: Michael Spagat et al., "Estimating War Deaths: An Arena of Contestation", Journal of Conflict Resolution 53, no. 6 (2009): 934-950. While individual conflict tolls can be––and are––contested, the overall decline in the deadliness of conflicts since the 1950s is not in doubt.
6. See chapter 2, footnote 4 in: Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010).
7. Emmanuela Gakidou, Margaret Hogan, Alan D. Lopez, “Adult Mortality: Time for a Reappraisal,” International Journal of Epidemiology 33, no. 4 (2004): 712.
8. Siyan Chen, Norman V. Loayza, and Marta Reynal-Querol, “The Aftermath of Civil War,” World Bank Economic Review 22, no. 1 (2008): 63-85.
9. See: Human Security Report Project, “HSRP Response to: Les Roberts’ Appendices A through C,” 14 April 2010, http://www.hsrgroup.org/images/stories/Documents/HSRP_Response_To_Dr_Roberts_Appendices.pdf (accessed 25 May 2010), 1. A more detailed discussion of our findings will be included in the published version of this report which will be released by Oxford University Press in the fall.
10. PRIO/UCDP have three categories of conflict: “Minor”, “Intermediate” and “War.” The review of African countries used the “intermediate” category; the global review used the “war” category.
11. For this review we used a more recent version of the IGME dataset that had been released after we had completed “Shrinking Costs.”
12. See: Human Security Report Project, “HSRP Response to: Les Roberts’ Appendices A through C,” 14 April 2010, http://www.hsrgroup.org/images/stories/Documents/HSRP_Response_To_Dr_Roberts_Appendices.pdf (accessed 25 May 2010), 1. A more detailed discussion of our findings will be presented in the upcoming published version of this report.
13. Les Roberts, “Commentary: Ensuring health statistics in conflict are evidence-based,” Conflict and Health 4, no. 10 (2010): 1.
14. Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 24. It is of course true that fatality estimates for many wars are subject to considerable uncertainty. But we know of no serious challenge to the central finding of the PRIO battle death dataset, namely that there has been a steep, uneven, but remarkable decline in the deadliness of warfare since 1950.
15. Les Roberts, “Commentary: Ensuring health statistics in conflict are evidence-based,” Conflict and Health 4, no. 10 (2010): 1.
16. See Chapter 2 of: Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010).
17. Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 18-20.
18. How the nature of warfare has changed and why this has reduced wartime fatalities is explained in Chapter 2. See: Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010).
19. Human Security Report Project, “The Shrinking Costs of War,” pre-publication text, 20 January 2010, http://www.humansecurityreport.info/2009Report/2009Report_Complete.pdf (accessed 25 May 2010), 2-4.
Competing interests
None declared