AI Literacy in the Context of Working with Sources

Pitfalls and Possibilities of Generative AI Models in Academic Writing

Abstract

This study examines the integration of generative AI (GenAI), such as ChatGPT, into students’ academic writing practices, focusing on its use for finding and working with sources. Using the concept of ‘imagined affordances’ we explore how students perceive and interact with this technology in academic contexts. We tested six student-centric prompting strategies across three fields using ChatGPT 3.5 and 4o, simulating realistic academic writing scenarios. Results show significant variations in the accuracy and usability of generated references across fields, strategies, and model versions. Notably, some strategies based on students’ imagined affordances, though technically unsound, produced useful outputs for academic writing tasks. ChatGPT 4o generally outperformed 3.5, highlighting rapid advancements in GenAI’s potential role in academic writing. These findings reveal a growing gap between institutional guidance on GenAI use in academic writing and students’ potential experiences. We advocate for a nuanced approach to AI literacy in higher education that acknowledges students’ perspectives, fosters open dialogue, destigmatizes experimentation while emphasizing critical evaluation, and raises awareness of how imagined affordances shape GenAI interactions during the writing process. This study contributes to discussions on AI integration in academic writing, offering insights for writing instructors, librarians, and policymakers.