Effects of Collaboration on the Performance of Interactive Theme Discovery Systems
NLP-assisted solutions to support qualitative data analysis have gained considerable traction. However, no unified evaluation framework exists which can account for the many different settings in which qualitative researchers may employ them.
@inproceedings{chen-etal-2026-effects,
abbr = {ACL},
preview = {inter_sys.png},
title = "Effects of Collaboration on the Performance of Interactive Theme Discovery Systems",
author = "Chen, Alvin Po-Chun and
Das, Rohan and
Srinivas, Dananjay and
Barry, Alexandra and
Seniw, Maksim and
Pacheco, Maria Leonor",
editor = "Liakata, Maria and
Moreira, Viviane P. and
Zhang, Jiajun and
Jurgens, David",
booktitle = "Proceedings of the 64th Annual Meeting of the {A}ssociation for {C}omputational {L}inguistics (Volume 1: Long Papers)",
month = jul,
year = "2026",
address = "San Diego, California, United States",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2026.acl-long.1968/",
doi = "10.18653/v1/2026.acl-long.1968",
pages = "42507--42526",
ISBN = "979-8-89176-390-6",
abstract = "NLP-assisted solutions to support qualitative data analysis have gained considerable traction. However, no unified evaluation framework exists which can account for the many different settings in which qualitative researchers may employ them. In this paper, we propose a framework to evaluate the way collaboration settings may produce different research outcomes across a variety of interactive systems. Specifically, we study the impact of synchronous vs. asynchronous collaboration using three different NLP-assisted qualitative research tools and present a comprehensive analysis of the differences in the consistency, cohesiveness, and correctness of their outcomes."
}

