Process guidelines for establishing Intercoder Reliability in qualitative studies

Qualitative interviews are increasingly being utilized within the context of intervention trials. While there is emerging assistance for conducting and reporting qualitative analysis, there are limited practical resources available for researchers engaging in a group coding process and interested in ensuring adequate Intercoder Reliability (ICR); the amount of agreement between two or more coders for the codes applied to qualitative text. Assessing the reliability of the coding helps establish the credibility of qualitative findings. We discuss our experience calculating ICR in the context of a behavioural HIV prevention trial for young women in South Africa which involves multiple rounds of longitudinal qualitative data collection. We document the steps that we took to improve ICR in this study, the challenges to improving ICR, and the value of the process to qualitative data analysis. As a result, we provide guidelines for other researchers to consider as they embark on large qualitative projects.