This study was performed in Mae Sot, Thailand among Burmese healthcare workers. The validation process is described for a Burmese and Karen language instrument. This includes: 1) study site description, 2) translation process; 3) determining the face and content validity; and 4) ensuring inter-rater reliability of the instrument.
Study site description
Study Site and Partners - The validation component was undertaken in August 2013 in the town of Mae Sot, Tak Province in western Thailand. Mae Sot is the major access point between Thailand and Burma and is within 3 km of the Burmese border town of Myawaddy. Mae Sot hosts not only a large number of Burmese migrants, but is also the headquarters of many international, national and community-based organizations that work with local communities in Burma. Several organizations operate health centers, mobile clinics, a hospital and transport services within Burma, while managing and providing administrative support from Mae Sot. JHSPH partnered with three health organizations based in Mae Sot during this study: Back Pack Health Workers Team (BPHWT), Burma Medical Association (BMA) and Karen Department of Health and Welfare (KDHW). BMA and KDHW consider themselves to be governmental health organizations in exile but all three groups are registered as non-governmental organizations in Thailand. The cross-border nature of the organizations’ work, in predominantly opposition held areas, is supported by local ethnic Burmese health workers who are able to work with local councils, camp and community leaders. This is key to their success and ability to reach areas historically inaccessible to humanitarian assistance. All organizations work to provide health care in accordance with principles of medical impartiality, although their actual or perceived affiliation to armed groups has increased their vulnerability [32].
All three organizations primarily serve communities in rural eastern Burma with basic healthcare services including primary care, vaccination, management of acute illness, simple surgical procedures, and reproductive health and midwifery services. The BPHWT serves 221,000 patients with over 95 mobile health teams that travel to remote and conflict regions to address local needs [28]. KDHW serves more than 100,000 internally displaced Burmese and BMA supports more than 40 clinics that serve 180,000 Burmese throughout eastern Burma. While the government of Burma now provides limited support for healthcare in this region, these organizations have been the primary providers of health services for a large and diverse population, particularly during the most volatile periods of eastern Burma’s ethnic conflict. These organizations were chosen for this study because they are the chief sources of medical care for communities in eastern Burma, expressed interest in improving their ability to collect data on attacks on their health workers and clinics, and through their headquarters in Mae Sot, are accessible to research institutions.
Study population - Health care workers from BPHWT were invited to participate in Phase 2 of the study, but staff from all three organizations actively took part in Phase 1. BPHWT workers with a wide range of skills and experience were chosen to participate in the validation because field experience during decades of chronic conflict ensured that participants have some knowledge of attacks and/or interference with health relevant to the instrument and the logistical and administrative structure of the biannual return of health workers to Mae Sot for training was well-suited to a timely and efficient study. BPHWT medics are community health workers trained in primary care and basic surgical techniques but do not have formal medical education. BPHWT field workers do not include health professionals such as nurses and doctors. The opportunity to partner with health providers that include staff with professional training was not possible due to the absence of such providers in this region and restrictions on access and security. Clinical and administrative supervisors at BPHWT headquarters identified study participants based on their field experience, willingness to discuss relevant topics and availability for discussion groups from among nearly 70 health workers who had returned from their primary mobile medical sites in Burma in August 2013. Suggested participants were requested to participate by their supervisors and informed that involvement was entirely voluntary and that they could decline without any consequences for their employment. Eligibility criteria required that participants be over 18 and medics with BPHWT.
Demographic characteristics - We held five discussion groups with group sizes of 6–9 aimed at ensuring meaningful conversation, with n = 38 ensuring saturation was met. Participants were 18 women and 20 men. Participants’ years of experience ranged between 0.5 and 10 years with a mean of 3.6 years of experience in their respective specialties. Forty-seven percent of participants were ethnic Karen but only 2 participants (5%) required the Karen translation of the instrument; all other participants used the Burmese form. The participants came from the following states: Karen, Rakhine, Shan, Kachin, and Kayan States. Participants included Field in-Charges (FiCs) [9], Maternal & Child Health Program health workers (MCH) [12], Medical Care Program health workers (MCP) [6], Community Health Education and Prevention Program workers (CHEPP) [7], and general Health Workers (HW) [4]. “Field in-Charges” are lead health workers in a major target area with one or more mobile health teams; their role is to manage health workers on the mobile health teams in their respective field area as well as to liaise with the administrative and programmatic staff in Mae Sot. MCH and MCP health workers provide maternal/child and primary care (six main diseases and war trauma injuries) respectively. CHEPP health workers provide preventative health services such as health education as well as clean water and sanitation systems to schools and communities and general health workers assist Field in-Charges and other workers with clinical duties.
Translation and back translation
The English version of the instrument developed in Phase 1 was translated into Burmese and Karen by native Burmese and Karen bilingual translators. These versions were then back-translated into English by translators who had not seen the original English version. The first author compared the back-translated copy to the original English version to identify incongruities. The Burmese and Karen translations were then adjusted with corrective re-translation if necessary.
Measures and analysis
This study utilized qualitative and quantitative evaluation methods to ensure robust testing of the instrument. The instrument was validated through face validity, content validity and inter-rater reliability.
Face and Content Validity - Face validity is the qualitative assessment that a survey reflects what it purports to measure [33]. Content validity measures whether the content of the tool is appropriate, relevant and correctly addresses the intention of the instrument [34]. Five discussion groups of 6–9 participants met with investigators for 4–4.5 hour sessions over three days. Upon conducting the study, saturation of ideas and responses was reached after 30 participants, but another discussion group was held to ensure completeness, leading to a total of 38 participants.
A trained Burmese or Karen translator with knowledge of human rights issues was present throughout all the sessions. The translator’s role was to directly translate investigator and participant comments and, as necessary, interpret the comments for better comprehension. Both translators worked previously in social justice organizations and had the relevant vocabulary and context to translate the content of the study process. Prior to the discussion groups, study investigators who had assisted with instrument development, and translators held briefings to review the content, language and goals of the study.
Item-by-item discussion was conducted utilizing a pre-written open-ended discussion guide. The discussion guide was structured to concentrate on the following key areas to determine face and content validity: 1) design of survey (layout, order, length); 2) language (translation, clarity, vocabulary, brevity and focus); 3) applicability and specificity of the items. Questions were asked about each domain, (i.e. attacks on health facility domain). Participants then were asked about each item within the domain, (i.e. impact on facility). Open-ended initial questions on each domain and each item were followed-up with more specific queries to clarify responses and probe any confusing issues.
Data analysis followed a constant comparative method for qualitative data categorization and analysis [35]. The first two authors participated in data analysis and categorization; the first author coded data, which was re-checked and discussed by members of the study team. Discussion notes were reviewed after each session and emerging themes and key points were documented.
Inter-rater reliability - Inter-rater reliability refers to the ability of different participants to consistently complete the instrument with the correct information given the same initial data [36]. The instrument was completed twice by participants using two different simulated incident scenarios and inter-rater reliability was assessed via scoring of participants’ completed reports. For authenticity, the scenarios were based on attacks and interferences similar to those previously documented in eastern Burma. Scenario 1 concerned a health worker who was beaten and had his supplies confiscated while traveling through the forest from one mobile clinic to another. Scenario 2 was the account of a medical clinic that was attacked, burned down and subsequently forced to close. Detailed information on dates, time, location and witnesses were provided for both scenarios. The translator presented the scenarios verbally and any questions were answered based on the script provided. Participants were allowed as much time as they required to complete the instrument. Subsequent to the completion of the simulated scenarios, participants were asked to provide feedback on their experience of completing the instrument.
For the simulated scenarios, time to completion was recorded for each group. Completed instruments were coded and graded for percent agreement compared with an answer sheet developed by the investigators. The completed instruments were evaluated under two criteria: 1) comprehension and reporting of the “essential facts” of the incident and 2) item validation wherein each item on the instrument was analyzed discretely. The “essential facts” were defined as the events of importance in the incident that would reveal who was involved, where and when the incident occurred and what happened, including the impact of the incident.
After the validation and analysis, the survey was finalized to remove any confusing language, correct translation errors, and improve the design and layout. The instrument is included as Additional file 1. Local health groups including BPHWT, KDHW, and BMA have versions in Burmese and Karen finalized for field use.
Human subjects protection
The Institutional Review Board (IRB) of JHSPH and a local review board convened in Mae Sot, Thailand, approved this study. To guarantee the confidentiality and security of participants, no individual identifiers were used. All participants provided informed consent prior to participation in the discussion groups, which were conducted by the first two authors in the presence of skilled local interpreters. Verbal consent was used to further safeguard participant privacy and security.