Skip to main content

Table 2 Recommendations for reporting validity information for the original population and implications when applying the tool to a new population of interest

From: Measuring mental health in humanitarian crises: a practitioner’s guide to validity

Reporting domain Validation information Population of interest Implications for interpretation
Population description Describe the population with whom the tool was previously validated (e.g., general community, persons with specific traumatic exposures, refugees, torture survivors) Describe characteristics of the new population being evaluated for humanitarian services If there are major differences in the original validation group and the new population evaluated in a humanitarian emergency, recommend caution for interpreting new population prevalence rates
Sampling strategy Describe how the validation sample was recruited (e.g., community recruitment, clinical facility recruitment) Describe how current population will be sampled for evaluation Convenience and clinical populations likely overestimate prevalence compared to the general population
Clinical characteristics Describe whether the validation sample was help-seeking, had any medical comorbidities (e.g., HIV, disability) Describe any pertinent clinical aspects of the population of interest Clinical characteristics may contribute to greater or fewer false positives, e.g., help-seeking populations may have fewer false positives than non-help-seeking groups
Demographic characteristics Describe age, gender distribution, relevant ethnic/ racial/ caste/ or other social groups, economic status Describe demographics Cut-off thresholds established with older or younger groups or racial/ethnic majority groups are likely different compared to other age groups or persons from marginalized groups
Language Report language of validation, clarify regional variants, specify literacy level (e.g., illiterate vs. college-educated population) Report language and literacy levels Tools validated with higher literacy groups may not be valid when applied to lower literacy groups; validity is impacted when moving between regional variants; if language variant or literacy level changes between original validation and current population, then consider additional qualitative evaluation
Validation tool Describe the gold standard tool used in previous validation Describe if any validation is done with the new population Particular gold standard tools may perform better in populations with regular healthcare access for ‘exclusion questions’, e.g., symptoms not attributable to a medical condition
Prevalence of mental health problem of interest Report the prevalence in the validation sample Report prevalence in the new sample The greater the difference in population prevalence between the validation sample and new population being evaluated, the greater likelihood of large numbers of false positives or false negatives
False positive rate Report specificity as well as standardized rates (e.g., ratio of false positives to true positives) Given false positive rate of original validation, report approximate number of false positives in new population The original validation psychometric properties can be used to provide adjusted prevalence estimates, e.g., providing an adjusted estimate for lower prevalence after accounting for false positive rates
False negative rate Report sensitivity as well as standardized rates (e.g., ratio of false negatives to true positives) Given false negative rate of original validation, report approximate number of false negatives in new population The original validation psychometric properties can be used to provide adjusted estimates and number of potential false positives