Social Pathology and Prevention 2023, 9(2):77-90 | DOI: 10.25142/spp.2023.019

WHEN TWO PEOPLE DO THE SAME THING, IT’S NOT THE SAME THING: CONSISTENCIES AND DIFFERENCES IN CONTENT ANALYSIS OF THERAPY DIARIES

Irena Loudová¹, Kateřina Strnadová², Silvie Urbanová²

¹ Department of Pedagogy and Psychology, Faculty of Education, University of Hradec Králové

² Institute of Primary, Pre-primary and Special Education, Faculty of Education, University of Hradec Králové

One of the frequent objections to qualitative approaches and the use of qualitative methods for research is the subjectivity of the evaluation – it is entirely up to the analyst to choose which sections to analyse, what codes to use, what themes to generate as significant and pass them on for further analysis or interpretation. This study, which is part of the broader DAQUAo project (psycholinguistic analysis of therapeutic diaries), is concerned with content analysis as a fundamental and usually the first step in most qualitative approaches. The aim of this study is to verify the agreement of assessors by statistically comparing codes obtained by the content analysis method in the open coding phase, performed by two independent analysts on identical texts. Stimulus material: therapy journals, N=179 unique text entries, total 22,046 words.

Results: Analytical K identified 32 codes, analyst L identified 70 codes. Codes that were close in content were referred to assessor agreement analysis (paired t-test). There was an agreement for two codes (movement-travel, emotion-good mood). Most of the differences were due to different levels of generality – there were mostly hyper- and hyponymy relationships between the analyst’s codes K and L (the superordinate category K corresponded to several specific categories of L) or adjacent categories (two specific instances of one superordinate category). Approximately one-third of the codes were unique and could not be easily compared.

Conclusion: coding text through content analysis requires careful training, a clear definition of the level of generality required in the first stage of coding, and last but not least, a formulated research question. These three conditions can ensure higher inter-assessor agreement and de facto better reliability of the method as such.

Keywords: Content analysis, therapeutic diaries, inter-assessor agreement

Received: June 19, 2023; Revised: October 16, 2023; Accepted: October 19, 2023; Published: May 24, 2024  Show citation