Measurement of attacks and interferences with health care in conflict: validation of an incident reporting tool for attacks on and interferences with health care in eastern Burma

Attacks on health workers and other types of interferences with healthcare services pose a considerable burden on communities and providers in conflict zones. The problem is receiving increased international attention, including publication of country-based case studies in very different contexts, such as Pakistan, Iraq, Afghanistan, Yemen and Nepal [1–8]. These studies highlight the types of attacks and interferences with healthcare including damage or destruction of health facilities, occupation and armed entry of premises, arrests, kidnappings and intimidation of health providers and patients, and attacks on and obstruction of health transports [9]. Armed conflict and other civil disturbances (which we refer to collectively as “conflict”) can lead to reductions to, or suspension of, vital medical services and denial of access to patients. Decreased access to vital medications and services can significantly impact the health of communities, particularly when the conflict and attacks on healthcare are long-standing [10]. This paper addresses the need for, and validation of, a tool that can be utilized by local health providers to report attacks toward the objective of greater protection and accountability, specifically in the setting of eastern Burma.

Local and international human rights organizations as well as UN agencies have provided valuable documentation on incidents in specific countries, but systematic tracking has been limited [11–17]. As a result, there exists a lack of data to address important questions concerning the dynamics of attacks including type of infrastructure at risk, victim profile, perpetrator profile and other potential sources of vulnerability such as time of day of incident, and mobile versus static services. Further, while most attention has been paid to attacks on international agencies, ICRC’s most recent global report found that local providers accounted for 91% of the 319 incidents, suggesting that development of a tool for use by local health providers could enable locally driven data collection to fill a significant gap in knowledge, inform protection strategies and provide a basis for accountability [18].

One country where attacks on health have been a chronic and severe problem is Burma, also known as Myanmar, where a conflict between ethnic-based armed groups and the ruling military junta has been ongoing for decades and has been characterized by major human rights abuses [19, 20]. Attacks on health services in the eastern states of Burma, and the far reaching impact on the health care system there are rarely reported on, except through the local human rights groups and organizations providing cross-border medical care. They have documented health workers beaten or jailed and patients halted at checkpoints and prevented from accessing care, among other violations of international humanitarian law (IHL) and international human rights law (IHRL) [21, 22]. These attacks have taken place against a background of governmental neglect of social and health services and especially poor health indicators among the ethnic minorities, particularly among the hill tribes along the mountainous Thai border of eastern Burma [23–26]. The maternal mortality rate is 721 per 100,000 live births in eastern Burma (covering the states of Karen, Mon and Shan, as well as two divisions - Bago and Thanintharyi), nearly three times the national rate of 240 [27]. Community-based health organizations in eastern Burma have sought to fill the gap in health services left by the government, but have been targeted for doing so. Though peace accords were signed with several armed groups in late 2011, hopes for genuine peace remain uncertain, and healthcare continues to be targeted in some regions [12].

Organizations providing health care in eastern Burma have sought a uniform and effective means for reporting these attacks. The aim of this study was to develop an easy-to-use instrument to enable more systematic reporting on such incidents. This context was identified as particularly suitable for instrument development, due to the partners’ commitment to documenting attacks and interferences with healthcare delivery as a component of their broader health systems data collection [22, 27, 28]. In developing such a tool, the authors anticipate that organized and pertinent information will permit organizations to identify and report attacks, as well as aid in prevention, protection and accountability. As victims and witnesses of attacks, local health providers and their staffs are ideally suited to act as the first line reporters of incidents. Educating local health providers to document their experiences on a standardized instrument could empower them to inform themselves of their rights and advocate on their own behalf.

The incident reporting form, (hereafter “the instrument”), was developed by a research team at the Center for Public Health and Human Rights at Johns Hopkins University, Bloomberg School of Public Health (JHSPH), as part of a multi-part project to address vulnerabilities of health in conflict. Phase 1 of the study included 1) a systematic literature review of recent peer-reviewed and grey literature [n = 20] documenting attacks on and interferences with healthcare occurring across conflict-affected countries [29] 2) a focused review of how international humanitarian law and human rights law bear on the problem [30] 3) qualitative in-depth interviews with supervisory health workers [n = 27] who have worked in conflict affected regions of eastern Burma [31] and 4) review of an early draft of a proposed instrument with key informants on the Thai/Burma border to further refine the instrument prior to validation (see Figure 1). The results of the first three parts of Phase 1 research were used by the research team to develop a simple but inclusive 33 item draft instrument. The items in the instrument are grouped according to four domains of violence against health care in conflict. These consist of attacks and interferences on 1) health care workers; 2) patients; 3) health care facilities and 4) health care transports. Items also cover the time, location and identity of the victims and perpetrators involved in the incident. Table 1 describes the key definitions and main domains of the instrument. Phase 2, reported on in this paper, aimed to determine the face and content validity and inter-rater reliability of the instrument. In subsequent phases, the research team plans to adapt the tool for use in other countries affected by violence against health services, and employ the technology of mobile devices to collect information on attacks on health services.

This study was performed in Mae Sot, Thailand among Burmese healthcare workers. The validation process is described for a Burmese and Karen language instrument. This includes: 1) study site description, 2) translation process; 3) determining the face and content validity; and 4) ensuring inter-rater reliability of the instrument.

This study validated a newly developed instrument to capture incidents of attacks or interference on health in eastern Burma. The findings of the qualitative and quantitative portions of the study suggest that the instrument is an effective and reliable instrument for health workers to report incidents of attack or interference with their daily work. Although limited to one setting, the instrument represents a first step toward more systematic reporting of attacks or interference with health and can provide a model for reporting attacks or interferences on health in other conflict regions.

Participants found the instrument relevant and applicable. They determined that the instrument appropriately reflects the intended outcome (face validity) and that it has an appropriate breadth and depth of information without being excessively cumbersome (content validity). Participants were able to comprehend the details of each domain but reported some unfamiliarity with healthcare transports in their own context. Despite this hurdle they were able to give examples of transports and discuss the importance of including a transport domain. Within each domain individual items were discussed and readily understood. Respondents did express the view that an element of subjectivity exists in interpreting some forms of attacks, such as interrogation and threats. After discussion, investigators simplified the numerical data fields to reduce error (from open to closed-ended answer choices) but retained the same categories (i.e. interrogation, threaten etc.) in the expectation that despite some subjectivity, the fields continue to provide a useful means of capturing those interferences that do not constitute violent attacks, but still represent important violation categories. One of the key goals of a standardized reporting tool is to develop some quantifiable data so it was important to retain the numerical fields while making them easier to use.

On evaluation of participant responses based on two scenarios, investigators found that participants were able to appropriately give the relevant information and provide insight into the extent of damage from an attack or interference with healthcare. Investigators agreed prior to field testing that 50% of participants accurately reporting the “essential facts” would assure that the instrument was valid based on the purposes for which the data collection would be used. This instrument is intended to provide an overall picture of the numbers and types of incidents, as well as a basis for further investigation to verify incidents of attacks or interference with healthcare. To determine whether an act constitutes a violation of international law, such that accountability procedures should be considered, more rigorous investigation and verification would be required. For this baseline data collection and reporting mechanism, however, some inaccuracy in reporting on incidents would be acceptable. The authors concluded that this level of accuracy in reporting is appropriate to develop a better understanding of the frequency of incidents, particularly non-violent interferences and their impact on health care delivery. This could inform protection strategies in health delivery and be used for national and international advocacy. The results that 86% of participants responding to Scenario 1 and 82% of participants responding to Scenario 2 reported “acceptable” accounts of the incidents indicates that the instrument is a reliable tool for reporting the numbers and types of attacks and interferences on healthcare in this context.

The time required to complete the tool was significantly less for Scenario 2 than for Scenario 1, and we predict that with further instruction and practice, the instrument will take less time to complete. Though respondents indicated that Scenario 2 was easier to complete, it is noted that more respondents scored “acceptable” on Scenario 1 (86% vs. 81%). The difference between these percentages is not statistically significant (p = .7). Nevertheless, this trend may be the result of the increased difficulty level of Scenario 2 as compared to Scenario 1 or other factors that have not been fully evaluated.

Item-by-item evaluation revealed that most items on the instrument were correctly completed. Investigators also observed that participants left some fields blank. One disadvantage of a closed-ended questionnaire is the possibility that respondents leave the response blank, making it unclear to investigators whether respondents did not understand the question, did not find an appropriate response within the closed-ended answer choices, or did not read through the survey thoroughly. Investigators have addressed this concern by revising the answer choices to include “unknown” fields among the answer choices, and the written instructions accompanying the survey set out that all fields should be completed.

On item-by-item validation, it was noted that only 21% of participants correctly completed the impact questions concerning treatment delay vs. interruption. The authors reviewed the impact questions closely and concluded that ascertaining the health impact requires additional data beyond the factual documentation of incident details. As the distinction between interruption of care and delay can be ambiguous (i.e. every interruption causes a delay), responding to these questions may be difficult. However, we expect that in other settings some additional information would be available and would reveal valuable information relevant to the impact of the incident. It is important to note that participants were guided through an item-by-item discussion of the instrument, but were not given formal training or definitions to aid completion of the instrument. Our findings suggest that training would assist in improving the reliability of responses and the comprehension of more difficult questions. We expect that the minor changes in the layout and design, and additional training prior to use in the field will better equip health workers in the future to complete the instrument in its entirety.

This validation process had several limitations. The instrument validation was conducted in one context on the Thai/Burmese border among Burmese health workers. Participants are qualified as “medics” by unofficial training programs and work experience. Many of the participants lack formal education beyond the grade school level and none have formal medical education. Conducting the study with formally trained clinicians in other settings may have yielded different results of the validation study. This may have biased the validation towards less trained personnel, rather than highly trained health workers such as physicians. Conversely, it is likely that in settings with more formally educated medical personnel the form will be filled out with similar if not greater accuracy.

As the instrument was translated from English into both Burmese and Karen, some language in both the reporting form may have been misunderstood. Investigators have attempted to minimize translation errors via verifying a back-translation and ensuring that discussion groups provided feedback on the translation.

Given the lack of healthcare transports used in this setting, investigators did not provide a scenario on health transport attacks. Though this omission may represent an incomplete validation of these items, the strong results in the other attack categories indicate that the transport section of the instrument, with similar answer fields, is reliable.

Responses and attitudes towards simulated scenarios in the safe environment in Mae Sot might be different from what might happen in the field, where stresses from insecurity could affect reporting. The majority of participants said that they felt safe using this form but also had thoughts on how to make the form less conspicuous in the field because possessing this data may pose some risk during crossing of checkpoints or raids of the clinics. Security is a major consideration in the utilization of this instrument and must be determined by individual organizations in collaboration with local health workers.

The validation was limited to participants chosen by administrators from one organization in a local context, potentially introducing bias based on their geographical, organizational or personal beliefs. However, participants were drawn from all areas of Burma where BPHWT works, and the responses revealed that diverse views were obtained from this study, limiting this bias. Monitoring and evaluation of day-to-day field use of this instrument, ideally, in diverse settings, would lead to improvements of the instrument.

Meetings were held with BMA, KDHW and BPHWT senior staff and data personnel to discuss the results and revisions to the instrument. The organizations indicated that they considered the form practical and relevant for their health workers to collect information in the field. Participants and administrators felt secure carrying the instrument for practical field use.

To our knowledge, there has been no validated incident reporting form for health workers and health organizations to report violations involving healthcare, particularly in eastern Burma.

It is expected that this instrument can serve as a model to be adapted, with changes based on differences in contexts, to other settings. Wide adoption of such an instrument could achieve greater local awareness of the incidence of attacks in specific locations, a global database of events, and the basis for action to prevent attacks and hold perpetrators accountable. The data collected by this instrument goes beyond typical security incident reporting forms, by capturing both violent and non-violent forms of interference with healthcare, such as confiscation of medicines, delays at checkpoints, obstruction of access, and intimidation and threats. Research in this setting indicates that such interferences are more frequent than violent events, and have negative consequences for access to and delivery of healthcare [32]. Increased reporting of non-violent events can assist health organizations in assessing the frequency and impact of incidents that health workers may have previously considered an ‘everyday event’. The reporting form is also tied to international human rights and humanitarian law. Creating an evidence base is crucial to an understanding of the role of non-violent interferences in curtailing health access and violations of the right to health in chronic conflict settings.

As frontline workers, human rights organizations, NGOs and others in conflict areas seek to improve the health of their communities. This reporting form has the potential to empower local health groups to understand their human rights and provide them with a simple effective means of reporting violations within their community and at the national and international level. Although this instrument was validated in the context of eastern Burma, it can provide a model for collecting data in other settings, ensuring a better evidence base from which to advance and advocate for the better protection and respect of healthcare in times of conflict.

In the setting of eastern Burma, investigators have developed an online platform using the MagPi website that partner Burmese organizations could adopt to collate their information [37]. The MagPi system allows for mobile phone entry and transmission of information as well as real-time data retrieval capabilities in a secure online database. Investigators expect that a mobile phone platform, which is not practicable at present in eastern Burma, will be of additional benefit in other settings where mobile data collection is more feasible. A global template that can be adapted for use in other contexts to document attacks on and interferences with healthcare can be accessed by contacting the corresponding author.