Should all qualitative researchers use LLMs?

One of the big topics at the AOM 2025 conference this summer was the use of large language models in the research process, especially in qualitative studies. We expand this discussion by asking: can qualitative research be automated—or augmented? Yes and no. Some of the advantages LLMs bring to the table are hard to ignore. LLMs can act as critical reviewers, as a consistency checker, as a provider of alternative perspectives on unstructured data, or to break path dependencies in the process of data analysis. They can also help find interesting outcomes that qualitative insights could explain. At the same time, the use of LLMs comes with thorny pitfalls. We know they are unreliable and hallucinate. And the output they create is… average at best. So if you use LLMs, make sure you are not using it for automation—do not lose touch with your craft or your data. Whatever tool you use, make sure you remain a virtuous scholar. Episode reading list Noblit, G. W., & Hare, R. D. (1988). Meta-Ethnography: Synthesising Qualitative Studies. Sage. Recker, J. (2021). Improving the State-Tracking Ability of Corona Dashboards. European Journal of Information Systems, 30(5), 476-495. Rynes, S., & Gephart Jr., R. P. (2004). Qualitative Research and the "Academy of Management Journal". Academy of Management Journal, 47(4), 454-462. Geertz, C. (1973). The Interpretation Of Cultures. Basic Books. Boland, R. J. (2001). The Tyranny of Space in Organizational Analysis. Information and Organization, 11(1), 3-23. Weber, R. (2004). Editor's Comments: The Rhetoric of Positivism Versus Interpretivism: A Personal View. MIS Quarterly, 28(1), iii-xii. Lehmann, J., Hukal, P., Recker, J., & Tumbas, S. (2025). Layering the Architecture of Digital Product Innovations: Firmware and Adapter Layers. Journal of the Association for Information Systems, 26, Lindberg, A., Berente, N., Howison, J., & Lyytinen, K. (2024). Discursive Modulation in Open Source Software: How Communities Shape Novelty and Complexity. MIS Quarterly, 48(4), 1395-1422. Ragin, C. C. (1987). The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. University of California Press. Goodhue, D. L., Lewis, W., & Thompson, R. L. (2012). Comparing PLS to Regression and LISREL: A Response to Marcoulides, Chin, and Saunders. MIS Quarterly, 36(3), 703-716. Goodhue, D. L., Lewis, W., & Thompson, R. L. (2007). Statistical Power in Analyzing Interaction Effects: Questioning the Advantage of PLS With Product Indicators. Information Systems Research, 18(2), 211-227.

You can listen to Should all qualitative researchers use LLMs? online on Radio and Podcast. Open the player on this page to stream the available audio.

Should all qualitative researchers use LLMs? is an episode from this IS research by Jan Recker.

This episode is 52:42 long.

This episode was published on Sep 9, 2025.

Yes. Use the heart button on the episode page to add it to your favorite episodes list.

Yes. This page shows related episodes from this IS research when more episodes are available from the podcast feed.