In fall 2006, a controversial estimate of Iraqi war deaths published in the Lancet [1] made world headlines, spurring a renewed round of debate over the ethics and consequences of the US-led Iraq invasion. The survey found that some 650,000 Iraqis were likely to have died as a result of the insurgency and sectarian strife following the 2003 invasion. The political ramifications of this figure were undeniable, given US leaders' insistence that their invasion had been, in part, motivated by humanitarian considerations [2]. Yet the report also became a magnet for critics, with many questioning the study's baseline assumptions, sampling methods, and data reporting procedures [3–6].

Methodological criticisms aside, the Lancet-inspired media furor clearly heralded the growing impact of public health research on conflict and human rights analysis. In particular, it drew attention to the capacity of "conflict epidemiologists" to provide science-based estimates of the direct and indirect cost of war. Most importantly, perhaps, these epidemiologists are gradually demonstrating that most existing studies grievously under-estimate war's overall human cost by failing to capture its indirect and long-term impacts [7–11]. From a human rights perspective, moreover, the legal liability of the commanders and politicians responsible for this collateral damage remains uncharted territory.

Epidemiological studies can also generate important evidence for policy decisions, as witnessed in the case of the Democratic Republic of Congo (DRC), where surveys by the International Rescue Committee (IRC) have called attention to the country's ongoing humanitarian crisis by discovering vast numbers of indirect, war-related deaths [12–15]. Of 3.9 million excess deaths from 1998 to 2004, according to these surveys, only a small proportion have been directly related to political violence, with the remainder attributed to war-related ailments, such as disease. These findings have proved influential in policy circles, boosting the conflict's international profile while enhancing the resources available to peacekeepers and aid workers alike [comments by representatives from the IRC, Human Rights Watch, and Catholic Relief Services, at workshop on "Integrating Public Health Methods and Data into Conflict Analysis," Ottawa, March 9, 2007].

Public health research has also helped assess harmful policy impacts short of armed conflict. One case in point is the 1999 study by a Columbia University researcher on the link between sanctions and Iraqi child mortality, which revised previous estimates downwards while still confirming that childhood mortality had risen at alarming rates [16]. These and other findings, according to one UN insider, played a key role in curbing the international body's appetite for comprehensive embargoes [comment by Andrew Mack (former head of strategic planning for the UN Secretary General), at workshop on "Integrating Public Health Methods and Data into Conflict Analysis," Ottawa, March 9, 2007].

Finally, consider another 1999 study of mortality, this time in the Serbian province of Kosovo, which argued that 12,000 people likely died during the conflict between Serbia, NATO and the Kosovo Liberation Army [17]. By mapping trend data against key political and military events, the report demonstrated that Serbian military activities, rather than NATO air strikes, were correlated with spikes in mortality. This study broke new ground by linking survey research to international humanitarian law (IHL, or the "laws of war"), and its findings have found their way into deliberations at the International Criminal Tribunal for the former Yugoslavia [18]. More than any other study, perhaps, this analysis illustrates the common interests of scientifically rigorous public health researchers, policy-oriented conflict analysts, and human rights monitors, underlining the value-added by multi-disciplinary research.

Although not all public health research is of equal quality, this article argues that closer collaboration is likely to continue to benefit epidemiologists, conflict analysts, and human rights monitors. Until now, public health's specialized methods, logistical complexities, and high costs have hindered multi-disciplinary research, and many non-specialists only dimly perceive opportunities for collaborative efforts. Many public health specialists, moreover, have a hard time demonstrating their value-added to lay audiences. This article seeks to bridge this gap by outlining epidemiology's utility for policy-makers, conflict analysts, and human rights monitors. For maximum effect, it should be read by non-epidemiologists in conjunction with a more technically oriented epidemiological primer [19, 20].

Epidemiology is the statistical study of the distribution of health events, outcomes and risk factors. This paper focuses on one particular research tool: retrospective population-based surveys. This emphasis is not intended to identify epidemiology exclusively with survey methods, but reflects surveys' particular utility for assessing conflict and human rights impacts. Although real-time and accurate surveillance data from health care facilities often provide the best measures of current conditions, such data are rarely available in crisis zones. Retrospective surveys, which ask people to recall health events during a specified time frame, are a good way of bridging this gap. Epidemiology has no monopoly on survey research, and human rights groups also use retrospective methods; most, however, do not rely on population-based techniques.

Since it is rarely possible to survey an entire population (as in a census), researchers typically question samples that are, in theory, representative of larger groups. When samples are well designed, their measured characteristics should be similar to those of the population from which they are drawn. A retrospective survey thus involves a standardized, structured questionnaire about past events, an accepted sampling method, and a statistical procedure for inferring about the general population from the sample's findings. (The appendix provides a more detailed discussion of survey methodology.)

Historically, the field of conflict epidemiology emerged from public health research in humanitarian emergencies [presentation by Bradley Woodruff, at workshop on "The Epidemiology of Complex Emergencies," Ottawa, March 8, 2007]. Beginning in the early 1970s, researchers increasingly realized that epidemiology could help devise needs-based policies and make humanitarian assistance more effective, and in the 1980s and 1990s, epidemiological findings were used to create general recommendations for improving the effectiveness of humanitarian assistance. As a result, practitioner manuals have increasingly included guidelines on using epidemiological methods in humanitarian assessments. In recent years, experts have developed minimum standards for humanitarian aid [21] while standardizing epidemiological methods for assessing key indicators [22].

As noted above, conflict epidemiology has also proved relevant to general foreign policy debates, particularly those pertaining to the creation, deployment and effectiveness of humanitarian, peacekeeping and peace-building interventions. The quantification of sickness and death is a concrete measurement of the quality of life (or lack thereof) in insecure zones, and is easily understood by most policymakers, even if the specific research methods are not.

Although epidemiology is only one of several quantitative methodologies which can and should be used in assessing conflict and human rights conditions, it has great potential due to its ability to offer detailed knowledge about what is happening to people in conflict, and the immediate causes of those events. It can establish numbers and rates of health events within populations, and, importantly, identify risk factors for such outcomes in specific times and places. In sufficiently large samples, moreover, these data can be broken down by age, gender, ethnicity, caste, or region.

Health data is often of "dual use," informing both evidence-based humanitarian programs [23], as well as general policy, media and advocacy purposes. Crucially, epidemiological findings can be particularly useful to human rights monitors, since specific health risk factors may also be violations of human rights law or the laws of war. With proper sampling methods, epidemiology can give monitors a more accurate sense of how widespread particular violations may be, and when trend data are available, researchers can correlate health outcomes with key political, legal, or military events.

Despite these clear advantages, population-based methods are rarely used by influential conflict analysis and human rights NGOs. Although the Boston-based Physicians for Human Rights (PHR) does a superb job of combining population-based surveys and human rights questions (see below), it is a small player in comparison to major NGOs such as Human Rights Watch (HRW) or the International Crisis Group (ICG). In 2005, according to annual reports, PHR's total budget was about US $4 million, compared to $11.4 and $26 million for the ICG and HRW, respectively. In 2006, moreover, PHR's 46 staffers were outnumbered by the ICG's 110 and HRW's 233. As a result of these discrepancies in size, PHR's comparative media impact is small. A keyword search of NGO names in the Factiva database, for example, found that in 2006, PHR was mentioned by The New York Times only seven times, compared to 63 and 157 for the ICG and HRW, respectively. These latter two NGOs are leading voices in global policy debates, and their research and advocacy is often considered "state of the art." To illustrate the value-added of collaborative research, we critically survey a small and non-random sample of HRW's and ICG's work.

HRW's and ICG's fieldwork is done at comparatively low cost, often with a slimmer field presence than epidemiological surveys would require. HRW typically sends a handful of researchers from its offices to record testimonies, often without explicit government permission. ICG staffers tend to be based more frequently within their countries of interest, but their research is similarly unobtrusive. Both groups rely on lengthy, unstructured interviews, but ICG's researchers focus more heavily on broader political and governance structures, while HRW concentrates on human rights violations. HRW generates new information on human rights abuses, but ICG sees its value-added as one of analysis and prescription.

Conflict epidemiologists are particularly concerned with conditions during and after complex emergencies, defined as "relatively acute situations affecting large civilian populations, usually involving a combination of war or civil strife, food shortages and population displacement, resulting in significant excess mortality" [34]. Mortality is a key indicator of overall population health [35], but epidemiologists may also seek information about a range of other health indicators, including morbidity, malnutrition, sanitation, and access to health care. Importantly, mortality is methodologically easier to assess than other health indicators. As noted above, mortality studies are important for assessing the direct and indirect impact of conflict, but this section also discusses other epidemiological research efforts relevant to conflict and human rights analysis.

Like any research method, retrospective surveys suffer from limitations, and they are neither useful nor appropriate for all times and places. For starters, population-based surveys are logistically complex and costly, requiring local teams of trained researchers, coordination and supervision. Epidemiology is a highly technical affair, requiring training and experience in sampling, questionnaire design, interviewing, and statistical analysis. Given these and other complexities, it is not surprising that experts often criticize field NGOs' surveys [51]. Keeping up with the methodological state of the art is difficult, and experts continue to refine accepted techniques. Experts, moreover, constantly debate the most appropriate methods for different settings [36, 52, 53]. Complex emergencies vary dramatically, and a one-size-fits-all research method is not appropriate [35].

Population-based surveys are often difficult to implement in insecure areas, since both survey teams and respondents are vulnerable and hard to monitor. Governments or armed groups frequently deny access, making studies difficult where they are needed the most. Importantly, surveys in politically tense environments can raise thorny ethical dilemmas by placing both informants and researchers at risk of reprisals or re-traumatization [54–58]. If epidemiology is increasingly used for human rights analysis and to provide grounds for external intervention, moreover, governments may begin to block general public health research among needy populations, to the detriment of humanitarian assistance programs. [personal email communication with Francesco Checchi, November 6, 2006]. Yet the failure to use powerful research methodologies for advocacy on behalf of vulnerable populations may itself be unethical [54].

The survey process is also vulnerable to political manipulation from all sides. For instance, asking respondents about who is responsible for individual deaths is problematic, as respondents may give false information for a wide variety of personal and political reasons. Respondents may also not be willing to tell interviewers that members of their households were combatants. Another ethical issue arises if everyone involved in a survey, including researchers employed by humanitarian agencies, has an interest in inflated numbers. For this reason, many experts believe that scientific data collection and political advocacy should be kept separate to maintain the science's legitimacy and credibility [participant comments at workshop on "Integrating Public Health Methods and Data into Conflict Analysis," Ottawa, March 9, 2007]. These issues should not preclude collaboration between epidemiologists and conflict analysts/human rights monitors, but they do need to be addressed in the research process.

A final drawback of epidemiological research is that the relevance of its findings can be difficult to convey to policy-makers and the general public. As the polemic inspired by the Iraq Lancet study suggests, the media's agenda may focus too heavily on perceived methodological problems, despite poor understanding of the technicalities involved, and of these problems' implications for the results' validity. Policy-makers opposed to a given study's findings will dismiss them as imprecise, while advocates may fail to acknowledge that their numbers come with biases and substantial margins of error.

While epidemiology is a powerful and under-exploited tool, the quantification of suffering is rarely sufficient, on its own, to ensure action. The political, economic, and logistical barriers to effective external intervention are well known, while new research has emerged suggesting that there are also substantial psychological barriers to promoting better public awareness of, and concern for, mass atrocities [59]. Full exploitation of epidemiology's potential will thus require close collaboration between public health analysts, conflict researchers, and human rights monitors.

There is little doubt that the research and writing styles of large NGOs, such as HRW and the ICG, offer important advantages, including unobtrusive research in insecure areas, and broadly accessible, easy-to-read reports. More importantly, perhaps, their detailed, confidential interviews with officials and other key informants can help establish causality in ways that statisticians find hard to emulate. Although epidemiology can demonstrate correlations, precise causal links are often more easily revealed through qualitative methods, such as "process tracing" of political decisions, chains of command, and actors' intentions, which "is fundamentally different from statistical analysis because it focuses on sequential processes within a particular [...] case, not on correlations across cases" [60].

While all organizations using data should understand and communicate its limitations, we are particularly concerned with the work of organizations generating new data, such as HRW. Like most human rights groups, HRW's ethos is grounded in international law, and most of its employees are not trained in epidemiology or other quantitative methods. With limited staff and a host of pressing demands, HRW finds it hard to prioritize discussions of careful sampling and data collection. Still, the group is constantly re-examining its research methods, and innovative collaborative efforts are already underway, including the group's report on abuses in Kosovo, in which data from 577 witnesses was coded and analyzed [61], and its report on Bangladeshi police forces, which charted the distribution of killings per population across police divisions [62].

Qualitative research groups such as HRW are not likely to transform themselves into survey outfits in the near future. Still, HRW and other qualitative research groups can and should become more conscious of their methodological limitations. Conflict epidemiologists, among others, can help generate more scientifically defensible evidence, and can also help clarify what the evidence shows, and what it does not. Although neither HRW nor the ICG have voiced interest in creating an in-house epidemiological capacity, both have expressed an interest in public health collaboration, including joint questionnaire design and better use of existing epidemiological results. At the same time, we discern growing interest among public health researchers in broader dissemination of their methods and data, and in working with others on the underlying causes of conflict and human rights abuse [34, 63, 64]. To be effective, these different research communities should become more literate in each other's lexicons, and engage in more frequent and respectful collaboration. The time for new research partnerships has arrived.

In this paper, we have provided a number of examples of public health research with proven relevance to conflict and human rights analysis. We conclude with a final collaborative scenario: the application of IHL analysis to the long-term human costs of war. At present, IHL offers little commentary on the legality of destroying the public infrastructure necessary for long-term health and human rights, preferring to concentrate on war's shorter-term and more immediate effects [65]. International lawyers find IHL's proportionality principle particularly hard to apply over time due to the intervention of complicating factors that make it hard to link cause and effect. Twelve months after a war's end, how much of a country's increased infant mortality could realistically – and legally – be attributed to wartime actions by combatants, as opposed to those of myriad other actors and events?

Given these complexities, human rights groups have hitherto preferred to focus on shorter time frames, where causality and legal responsibility are easier to establish. Over time, however, the accumulation of good quality epidemiological data can help broaden and extend the IHL analysis to longer post-conflict periods. The availability of relevant information is crucial, since IHL violations are judged on expected losses weighed against anticipated military advantages. As one analysis notes, "it is unacceptable for the expected military advantage to be based on a longer timeframe while limiting the expected quantification of civilian damage only to the immediate effects of the attack itself" [66]. By repeatedly documenting the short, medium and long-term impact of specific military tactics, epidemiological research can force military planners to increase the horizon of what they can reasonably predict. This argument is already being used in the ongoing debate over cluster munitions, where some believe IHL requires commanders to consider the explosives' long-term threat to civilians [66].

Epidemiology is able to provide evidence of human suffering of great value to conflict analysts and human rights monitors. More often than not, information on the civilian impacts of conflict is based on informed guesses by NGOs and multilateral organizations, rather than rigorously assembled scientific data. This paper has identified problems of data use and collection by two major advocacy NGOs, arguing that these short-comings are particularly problematic when establishing the overall prevalence of a particular human rights abuse or conflict pattern. These data gaps, we argue, can be addressed in part through greater collaboration with public health researchers.

Epidemiology can help quantify the differential direct and indirect impact of conflict on particular populations, while trend data can track impacts over time, enabling researchers to map health outcomes against major offensives; peacekeeping operations; humanitarian assistance flows; and peace agreements. This information can shed light on the efficacy of international engagement in conflict zones, while providing human rights investigators with a way of assessing the extent and impact of violations across populations.

Research collaboration between public health specialists, conflict analysts and human rights monitors faces practical and ethical difficulties. These should be acknowledged and addressed, but they should not preclude the kind of collaborative research that could benefit needy and distressed populations.

Epidemiological surveys collect quantitative health indicators from populations at a specific time, using standardized, structured questionnaires. In retrospective surveys, surveyors ask respondents to recall health events that occurred during a specified time frame known as the recall period. Although surveys can be exhaustive by including every person in the population (such as a census), they are usually based on representative samples. When samples are well designed, their measured characteristics should be similar to those of the population from which they are drawn. Survey design has to contend with bias (non-sampling error) and imprecision (sampling error).

Sampling is the selection of a specified number of persons or households from a population. Epidemiologists usually employ probability sampling, which ensures that every selected person or household has the same known chance of inclusion. Sample size should be large enough to provide reliable estimates, but not so large so as to waste limited time and resources. Larger samples are required for greater statistical precision or to investigate a condition with low prevalence within the population, such as maternal mortality. There are three general methods of probability sampling.

Simple random sampling requires a complete list of all the units to be sampled, such as households, and a certain number are then randomly selected from this sampling frame. Although this method is often the most representative, it is rarely feasible in conflict settings because of the paucity of complete lists. But even if good listings are available, simple random sampling is generally more expensive as it requires broad coverage of wide areas, and is thus logistically complex. For these reasons, simple random sampling is often only used when studying registered populations that are concentrated in small areas, such as well-organized refugee camps.

Systematic sampling, by contrast, randomly selects only the starting unit; all other units are selected by adding a certain number (known as the sampling step), which depends on the desired sample size. While this method does not require a comprehensive list to start, it does need a well-ordered population and a good estimate of population size, so that the sampling step can be calculated and applied. Again, systematic sampling is often possible in refugee camps or other well-delineated populations.

A third method, multi-stage cluster sampling, begins by listing clusters of sampling units, such as administrative divisions or villages, and then randomly selects a certain number of these. Cluster selection must be proportional to relative population size, so that areas with greater populations are allocated more clusters. Clusters can be selected at more than one stage of sampling. At the final stage, variants of the sampling methods outlined above can be used to select an equal number of households from each cluster. In many cases, the first household is selected randomly, while the rest are selected by proximity to the first. This method is a good way of creating representative samples even when there are no adequate listings of the entire population, or when households are not distributed in an ordered pattern. To do a good multi-stage cluster sample, one must simply be familiar with basic geographic divisions and their relative population size. Cluster sampling may also limit logistical and security concerns by reducing the movement of survey teams to a few random points. This also makes cluster sampling cheaper than random sampling. For these reasons, cluster sampling is often the sampling method of choice in complex emergencies.

Multi-stage cluster sampling has important drawbacks, however. Cluster sampling cannot be used to analyze quantitative differences between geographic divisions unless the population is first stratified by relevant criteria, with separate cluster samples drawn from each stratum. This increases the overall number of clusters needed. Moreover, statistical precision is lower in cluster samples, since households within clusters are more likely to resemble each other than if they were selected randomly from the entire population. This leads to a loss in sampling variability known as the design effect. This is particularly problematic when measuring highly clustered phenomena such as the effects of violent conflict. To compensate, researchers must increase the sample's overall size. Since it is statistically preferable to increase the number of clusters rather than the number of households within clusters, this compensatory adjustment often boosts the survey's cost and duration.

Samples always come with biases, which should be minimized and acknowledged. To prevent avoidable biases, researchers must try to ensure that the data they collect closely reflects the respondents' situation. This requires that the data collection effort be standardized and tightly monitored for quality. Questionnaires should be simple and clear, and fewer questions generally provide better measurements. Survey interviewers should be identically trained so that they do not influence responses.