Implementation of Audio-Computer Assisted Self-Interview (ACASI) among adolescent girls in humanitarian settings: feasibility, acceptability, and lessons learned
© The Author(s). 2016
Received: 22 March 2016
Accepted: 14 September 2016
Published: 4 January 2017
Audio-Computer Assisted Self- Interview (ACASI) is a method of data collection in which participants listen to pre-recorded questions through headphones and respond to questions by selecting their answers on a touch screen or keypad, and is seen as advantageous for gathering data on sensitive topics such as experiences of violence. This paper seeks to explore the feasibility and acceptability of using ACASI with adolescent girls and to document the implementation of such an approach in two humanitarian settings: conflict-affected communities in eastern Democratic Republic of Congo (DRC) and refugee camps along the Sudan-Ethiopia border.
This paper evaluates the feasibility and acceptability of implementing ACASI, based on the experiences of using this tool in baseline data collections for COMPASS (Creating Opportunities through Mentorship, Parental involvement, and Safe Spaces) impact evaluations in DRC (N = 868) and Ethiopia (N = 919) among adolescent girls. Descriptive statistics and logistic regression models were generated to examine associations between understanding of the survey and selected demographics in both countries.
Overall, nearly 90% of girls in the DRC felt that the questions were easy to understand as compared to approximately 75% in Ethiopia. Level of education, but not age, was associated with understanding of the survey in both countries.
Financial and time investment to ready ACASI was substantial in order to properly contextualize the approach to these specific humanitarian settings, including piloting of images, language assessments, and checking both written translations and corresponding verbal recordings. Despite challenges, we conclude that ACASI proved feasible and acceptable to participants and to data collection teams in two diverse humanitarian settings.
Audio-Computer Assisted Self- Interview (ACASI) is a method of data collection in which participants listen to pre-recorded questions through headphones and respond to questions by selecting their answers on a touch screen or keypad . ACASI is believed to improve quality data collection, above and beyond general benefits of technological approaches to data collection which can minimize data entry errors and reduce time to clean and monitor incoming data . Such benefits of using ACASI may include minimizing skipping of questions and increasing reporting of sensitive behaviors [3–6]. The use of ACASI among adolescents in low- and middle-income countries has been growing in recent years. Topics have typically centered on sexual health, drug use, sexual risk-taking, or other sensitive behaviors [4, 7] due to ACASI’s potential to reduce the social desirability bias that may occur in face-to-face interviews . However, some criticisms of ACASI suggest the administration format may eliminate the empathy, trust and rapport that face-to-face interviews provide, which may provide other opportunities for disclosure [9–12] or outreach for referral services. The appropriateness of technological approaches like ACASI to elicit sensitive information like sexual behavior from very young adolescents has also been noted . While ACASI is increasing in popularity as a data collection method in development settings, little is known about its use in humanitarian emergencies. In particular, settings with refugees and/or internally displaced persons can be challenging environments for survey administration given levels of mobility and the fact that large, heterogeneous groups of people with various cultures and languages which often reside in close geographic proximity. Such heterogeneity may require surveys to be administered in multiple languages or require cultural adaptations that may increase the potential for measurement error or require substantial updating of the tools in the field. Extensive efforts for rigorous electronic survey administration of sensitive information may be difficult in such settings. In addition, there may also be challenges directly related to the use of ACASI, including concerns regarding risk of theft of portable technological devices  against the backdrop of limited or interrupted internet connectivity and electricity.
Thus, the current paper seeks to explore the feasibility and acceptability of using ACASI with adolescent girls and to document the implementation of such an approach in two humanitarian settings: conflict-affected communities in eastern Democratic Republic of Congo (DRC) and refugee camps along the Sudan-Ethiopia border. Specifically, this paper draws from experiences related to the baseline data collection for the COMPASS (Creating Opportunities through Mentorship, Parental involvement, and Safe Spaces) impact evaluations in DRC and Ethiopia. COMPASS is a comprehensive program that delivers a curriculum for adolescent girls in safe spaces and engages community members, service providers, and caregivers to create a supportive environment for girls to protect them from violence and encourage a healthy transition to adulthood. Primary outcomes of the COMPASS study include changes in sexual violence, unhealthy personal relationships, and early/forced marriage, among other gender-related outcomes. The primary objective of the present manuscript is to outline the preparation process for using ACASI in these settings, the technological specifications of the tools, and lessons from its implementation.
COMPASS, a multi-country research study, is led by Columbia University, in partnership with the implementing organization, the International Rescue Committee (IRC). In DRC, the baseline for the cluster-randomized controlled wait-list trial was implemented between May-July 2015. The overarching objective of this evaluation is to understand the incremental effectiveness of the parental engagement activities in addition to the core COMPASS components on girls’ experiences of violence and well-being compared to girls whose parents have not yet received the parental activities. In Ethiopia, the baseline for the cluster-randomized controlled trial was implemented between August-September 2015, which seeks to understand the overall impact of the COMPASS program on girls’ experiences of violence and well-being compared to the wait-list group. Girls aged 10–14 years in DRC and 13–19 years in refugee camps in Ethiopia were asked to participate in the study and complete a baseline survey. In total, 869 girls from DRC and 919 girls from Ethiopia completed the survey. All girls that had enrolled in the program were eligible for the study, and girls were formed into program groups based on geographic proximity and language. Groups were then randomly assigned to either the intervention group or the control group. Additional study details are found elsewhere .
Preparation of ACASI
The following subsections document the process of developing and using the ACASI technology, with particular focus on language and translation, use of visuals, and technical specifications.
Languages & translation
The original intention behind selecting ACASI was to improve the quality of reporting of sensitive experiences, such as forced sex and sexual history, which in Ethiopia was asked to all girls (13–19 years) and in the DRC was asked only to older girls in the study population between 13–14 years of age. In DRC, ACASI was selected to complement the CAPI (Computer-Assisted Personal Interview) sections of the survey which included less sensitive questions around demographics or gender norms, for example, and was administered by trained female data collection staff. Interview tools were translated and back-translated into Mashi and Swahili, the languages of survey administration. Final translations were checked with groups of girls to ensure the language was comprehensible for the target groups. Following this final check, audio files of questions and text were programmed into the tablets. Changes were made as needed during the baseline training.
In refugee camps in Ethiopia, the survey was originally to be implemented in Arabic, which is the language generally spoken in markets or community spaces in the refugee camps. However, formative assessments with girls revealed that understanding was not sufficient for survey implementation. Subsequently, a language assessment was conducted by IRC’s research and program teams which identified 19 tribal languages in the camps. Four of the most prevalent languages were selected for the survey to meet sample size requirements: Meban, Regarig, Ingessena Kulelek and Funj/Berta, which are all non-written languages spoken in Sudan. It was decided that the research would be conducted solely using ACASI, because the languages were non-written and low literacy levels made it unlikely that simultaneous translation by female enumerators from English or Arabic into a local language would be feasible. Refugee women were hired as ACASI assistants to set girls up with the tablets and demonstrate how to use it as they were able to communicate with girls in the local languages.
Translation therefore followed a careful process with a large team: male IRC community workers with adequate levels of literacy verbally translated the questions from English to the target language, and discussed the meaning and wording with a group of 2–3 non-literate refugee women and a native English speaker. One woman then audio recorded the question from memory after phrasing was agreed upon. This was then used as the basis for back-translation and reconciliation of any discrepancies between the original English questions. Final recordings were checked with girls to ensure they had a strong understanding of the questions. Once audio files were programmed into the tablets, the recordings were checked and amended a final time as needed during baseline training and piloting.
Response categories & images
Given contextual differences in colors or potential images, ACASI response option colors and images were piloted with girls in DRC and Ethiopia. In DRC, only six colors could be consistently named by all girls from these communities when tested which would have limited the potential response options in the ACASI tablet. Girls were not able to differentiate between shades of a single color and translation challenges made it difficult to describe shades of color for ACASI response options. Girls who had not been to school found recognizing colors particularly challenging; common fruits and vegetables were easier for them, though caution had to be exercised in the selection of images (there are multiple words for different types of banana in some languages, for example). While ACASI options had been programmed in local languages, it was found that individuals who had been to school were mainly familiar with the words for various colors in French (as taught in school) and not in the local language they spoke. Girls were given a sheet during survey administration that showed the response options and the corresponding words in their local languages. Testing found that a mixture of identifiable fruits and colors was the most appropriate way to have eight recognizable and distinct options for the target group in DRC (see Fig. 1). Previous research has documented that although some specifications of graphics or images (e.g., size; brightness of color) may influence study responses, they have not been fully examined in technological approaches to data collection, such as ACASI or CAPI . In Ethiopia, it was found the ten colors could be used in the ACASI screen for responses; thus no images such as fruits or vegetables were needed for response categories.
The present study used Android-based tablets which were programmed in New York City by the Population Council. The surveys on the tablets were updated as needed when wireless internet was available in country. Data from tablets was uploaded to a laptop computer in each country and exported to the research team via a secure cloud-based system. Further technological specifications are found in the Additional file 1.
Training data collection staff for administering ACASI
In Ethiopia, most of the ACASI assistants supporting the girls had very low levels of literacy and were also new to touch-screen technology; training was therefore extremely important in preparing them for data collection. Program staff assisted potential ACASI assistants with low English literacy in writing resumes and cover letters that could be submitted as an application of interest. Training occurred in phases: a basic introduction to handling the tablets progressed to practice with listening to the audio and selecting relevant responses, and then to interview simulations where ACASI assistants explained tablet use to each other, using the audio as a reference for practice question content. Lastly, the team conducted a pilot with 10 volunteers from one of the camps. Translators in the 4 languages used during data collection were present during training, and refresher trainings were conducted prior to launching data collection at subsequent camps after the first camp’s data collection was completed. Using young women as data collection staff from target communities proved valuable; their similarity to the girls in terms of background and age were important for their ability to guide the girls through the survey process. The training in DRC followed a similar pattern and piloting of tablets with girls and caregivers in communities. In terms of data collection staff themselves, skills were assessed in the initial training to gauge their comfort and understanding of ACASI in both countries. In Ethiopia, ACASI assistants who had very low levels of literacy were matched with ACASI assistants who had a better understanding of tablets from other camps for additional support throughout data collection. DRC data collectors were literate in French and either Mashi and/or Swahili (languages of survey administration).
At the conclusion of the survey, one question was asked to assess the acceptability and perceptions of the survey and using ACASI: “Do you feel the questions were easy to understand?”. Responses were based on a Likert scale from very easy to understand to very difficult to understand. A subsequent fill-in question posed to the participants was completed by the data collection staff: “What did you think of using the tablet?” in DRC only. This was asked only in DRC given the non-written nature of languages, low literacy levels of ACASI assistants in written languages such as English or Arabic, and limited availability of literate staff to write in responses while overseeing the data collection in the refugee camps in Ethiopia.
Descriptive frequencies were tabulated separately based on selected demographics for each country for the quantitative question using STATA. Wald chi-square tests were used to assess statistically significant differences in responses by demographics. Where appropriate, odds ratios were constructed in logistic regression models between demographics and responses related to survey understanding. In regards to the fill-in question, results were categorized into the following: (1) fine, comfortable happy; (2) very good; (3) enjoyed using the tablet;(4) very easy; (5) a little difficult; (6) very difficult; (7) fear; (8) tired; or (9) response unclear.
Frequencies and unadjusted odds ratios of demographics and understanding of survey
N = 868
10-12 (N = 491)
13-14 (N = 377)
Have ever attended school (N = 691)
Have never attended school (N = 178)
Completed primary as highest level of education (N = 637)
Completed secondary as highest level of education (N = 53)
N = 917
13-15 (N = 704)
16-19 (N = 205)
Have ever attended school (N = 636)
Have never attended school (N = 252)
Completed primary as highest level of education in Sudan/South Sudan (N = 615)
Completed secondary as highest level of education in Sudan/South Sudan (N = 9)
Completed primary as highest level of education in Ethiopia (N = 615)
Completed secondary as highest level of education in Ethiopia (N = 7)
In terms of fill-in responses regarding what girls thought of using the tablet, an overwhelming 97% of girls in DRC had a positive reaction. Nearly 65% of responses were categorized as fine, comfortable, or happy; 19% of girls reported that the tablet was very easy; and 7% and 5%, respectively, found the tablet very good or enjoyed using the tablet. Three percent of girls found the tablet a little difficult or very difficult to use, with a few reporting that they were a little afraid of it. No fill in responses were included in the Ethiopian survey given the non-written nature of languages.
The use of electronic data collection via ACASI systems to collect data from young adolescent girls on sensitive behaviors such as experiences of violence and sexual history is a new phenomenon in humanitarian settings. Findings of the baseline data collection for the COMPASS trials demonstrate that while overall understanding of the survey was acceptable, girls who had more education demonstrated higher likelihood of understanding. Additional time may be needed for researchers to give instructions on how to use the tablets to girls with lower educational attainment. Concerns of the appropriateness of ‘high-tech’ strategies have been noted elsewhere for data collection with younger adolescents . Our results show that these strategies are feasible and understanding does not vary by age, but rather by educational level. Additionally, other forms of data collection such as engaging or culturally specific and empathic face-to-face quantitative interviewing techniques have also been used for adolescents regarding sensitive behaviors . However, given the vast heterogeneity of ethnicities and languages found in our study sample, particularly among refugee camps in Ethiopia, this technological approach aided in survey administration. The use of ACASI also ensured anonymity and confidentiality of girls’ reported experiences in violence. Thus, all study respondents received a debrief script and referrals after survey administration, regardless of whether girls reported violence or not.
Additional lessons learned from implementing ACASI technologies are described below.
Scoping of wireless connectivity and electricity availability in each setting was helpful to adjust study implementation scheduling as needed to ensure adequate updating of tools and transfer of data as needed. The tablets had sufficient battery to last a full day, and teams were generally able to charge overnight, or rely on back-up tablets. However, in DRC, some teams experienced difficulty in finding a reliable source of electricity and were therefore unable to charge tablets in full in order to complete all scheduled interviews during the following day. Each team had one back up tablet and at times a nearby team was able to send over their back up tablet in a study designated vehicle to assist with completion of interviews. Interview schedules had to be adapted to account for partially charged tablets.
Risk of theft
The use of tablets in communities with little exposure to computerized technology drew attention from other community members, increasing the vulnerability of theft. During data collection in the DRC, five tablets stored in one bag were stolen out of a study vehicle. The team had backed-up data to a master tablet on a daily basis, but unfortunately this tablet was stolen too, resulting in the loss of 19 completed (de-identified) surveys. Lessons learned suggest that if data is backed up during the week on a different master tablet that such tablets are kept in separate areas.
Observations from data collection staff
When demonstrating the use of tablets and ACASI to girls, some survey staff reported that a small number of girls were worried about touching the screen because they were concerned they might break it and would be responsible for replacing the tablet in DRC. Others tried to flip the screen like pages of a book or responded to questions by saying their answer out loud rather than touching a screen. Survey administrators remained near the girl while maintaining her privacy so that they could quickly respond to concerns or questions. In DRC, some supervisors and data collectors began to do individual and group introduction sessions to the tablet to address any concerns before beginning the survey. Allowing girls to simply touch the tablet and headphones and see other girls doing the survey improved comfort with the tablets. Headphones for ACASI were also cleaned with a wipe in front of girls before asking them to put on the ear pieces. In Ethiopia, the ACASI assistants remained near the girls who were completing the survey to assist in any questions.
Interpretations of the process of implementing ACASI and lessons learned should be considered with limitations in mind. First, a comparison between the use of ACASI versus other forms of interviewing such as face-to-face paper and pencil surveys would have been useful to discern differences in reporting of sensitive violence experiences. In addition, further qualitative data collection after the interviews were completed to gain further insight into the acceptability of ACASI would strengthen the lessons learned. There is limited published research available regarding violence experiences among adolescent girls in humanitarian settings; thus, comparison of perceptions of the feasibility and acceptability of ACASI and tablet-administration is limited.
In addition, a small number of girls sought referrals following interviews. One potential limitation, although untested, is that this number of girls seeking referrals could be lower if compared to a situation where trust and rapport was built with an in-person interviewer. Additional data to examine the potential differences between interview formats in terms of referral patterns or other potential differences in psychosocial wellbeing following such a sensitive interview would be useful to include in future research.
Based on these baseline data collection experiences, ACASI proved feasible and acceptable to participants and to data collection teams in two diverse humanitarian settings. Admittedly, financial and time investments to ready ACASI were substantial in order to properly contextualize the approach through piloting of images, language assessments, and checking both written translations and corresponding verbal recordings. In refugee camps in Ethiopia, it was particularly useful given the numerous non-written languages of the study population. Despite these challenges, ACASI was successfully implemented in order to collect sensitive data among adolescent girls as part of the COMPASS trials.
The authors would like to thank collaborators from Columbia University, IRC Ethiopia and IRC DRC and IRC headquarters. In particular, we would like to thank Amy Neiman, Cécile Fanton d’Anton and Teame Tesfay for their support to data collection. We are also grateful to the girls and caregivers who are participating in the COMPASS research program.
This document is an output from a project funded with UK aid from the UK Department for International Development (DFID) for the benefit of developing countries. However, the views expressed and information contained in it are not necessarily those of or endorsed by DFID, which can accept no responsibility for such views or information or for any reliance placed on them.
KF(co-PI), KR, AA, and LS (PI) conceptualized the study. ST, KR, AA, WT, TB, and PM led study implementation and management. KA provided data analysis and SS and SM provided technical support for data collection. KF, ST, and KA led drafting of the paper. All authors have reviewed and approved of the final manuscript.
The authors declare they have no competing interests.
Ethics approval and consent to participate
All study protocols were approved by Columbia University (IRB-AAAP6855 & IRB-AAAO6612). Local approval was received by the Ministry of Gender in DRC and by the Administration for Refugees and Returnee Affairs in Ethiopia. Additionally, the study in Ethiopia also received approval from the IRC internal Institutional Review Board (WPE 1.00.003). All girls gave assent to complete the survey and caregivers provided informed consent for the girl’s participation if the girl was under 18.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Audio Computer-Assisted Self-Interviewing (ACASI) [http://www.popcouncil.org/research/audio-computer-assisted-self-interviewing-acasi]. Accessed 18 July 2016.
- Langhaug LF, Sherr L, Cowan FM. How to improve the validity of sexual behaviour reporting: systematic review of questionnaire delivery modes in developing countries. Trop Med Int Health. 2010;15(3):362–81.View ArticlePubMedPubMed CentralGoogle Scholar
- Turner CF, Ku L, Rogers SM, Lindberg LD, Pleck JH, Sonenstein FL. Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. Science. 1998;280:867–73.View ArticlePubMedGoogle Scholar
- Hewett PC, Mensch BS, Erulkar AS. Consistency in the reporting of sexual behaviour by adolescent girls in Kenya: a comparison of interviewing methods. Sex Transm Infect. 2004;80(S2):ii43–8.PubMedPubMed CentralGoogle Scholar
- Azevedo Simoes A, Inacio Bastos F, Ismerio Moreira R, Lynch KG, Metzger DS. Acceptability of audio computer-assisted self-interview (ACASI) among substance abusers seeking treatment in Rio de Janeiro, Brazil. Drug Alcohol Depend. 2006;82(S1):S103–7.View ArticleGoogle Scholar
- Hewett PC, Mensch BS, de A Ribeiro MCS, Jones HE, Lippman SA, Montgomery MR, van de Wijgert JHHM. Using sexually transmitted infection biomarkers to validate reporting of sexual behavior within a randomized, experimental evaluation of interviewing methods. Am J Epidemiol. 2008;168(2):202–11.View ArticlePubMedPubMed CentralGoogle Scholar
- Langhaug LF, Cheung YB, Pascoe SJS, Chirawu P, Woelk G, Hayes RJ, Cowan FM. How you ask really matters: randomised comparison of four sexual behaviour questionnaire delivery modes in Zimbabwean youth. Sex Transm Infect. 2010;87:165–73.View ArticlePubMedGoogle Scholar
- Metzger DS, Koblin B, Turner C, Navaline H, Valenti F, Holte S, Gross M, Sheon A, Miller H, Cooley P, et al. Randomized controlled trial of audio computer-assisted self-interviewing: utility and acceptability in longitudinal studies. Am J Epidemiol. 2000;152(2):99–106.View ArticlePubMedGoogle Scholar
- Poulin M. Reporting on first sexual experience: the importance of interviewer-respondent interaction. Demogr Res. 2010;22(11):237–88.View ArticlePubMedPubMed CentralGoogle Scholar
- Jaya, Hindin MJ, Ahmed S. Differences in young people’s reports of sexual behaviors according to interview methodology: a randomized trial in India. Am J Public Health. 2008;98(1):169–74.View ArticlePubMedPubMed CentralGoogle Scholar
- Estes LJ, Lloyd LE, Teti M, Raja S, Bowleg L, Allgood KL, Glick N. Perceptions of audio computer-assisted self-interviewing (ACASI) among women in an HIV-positive prevention program. PLoS One. 2010;5(2):e9149.View ArticlePubMedPubMed CentralGoogle Scholar
- Jennings TE, Lucenko BA, Malow RM, Devieux JG. Audio-CASI vs interview method of administration of an HIV/STD risk of exposure screening instrument for teenagers. Int J STD AIDS. 2008;13(11):781–4.View ArticleGoogle Scholar
- World Health Organization. The sexual and reproductive health of young adolescents in developing countries: reviewing the evidence, identifying research gaps, and moving the agenda. In: Geneva: WHO; 2011.Google Scholar
- Group NMHSPTfAAC. Designing an audio-computer assisted self-interview system in a multisite trial: a brief report. J Acquir Immune Defic Syndr. 2008;49(S1):S52–8.Google Scholar
- Falb KL, Tanner S, Ward L, Erksine D, Noble E, Assazenew A, Bakomere T, Lowry C, Graybill E, Mallinga P, et al. Creating Opportunities thorugh Mentorship, Parental involvement, and Safe Spaces (COMPASS) program: Multi-country study protocol to protect girls from violence in humanitarian settings. BMC Public Health. 2016;16(231):1–10. doi:10.1186/s12889-016-2894-3.
- Schaeffer NC, Dykema J. Quesitons for surveys: current trends and future directions. Public Opin Q. 2011;75(5):909–61.View ArticlePubMedPubMed CentralGoogle Scholar