Accurately assessing students’ use of generative AI acknowledgements in assignments

Dr Lynette Pretorius

Contact details

Dr Lynette Pretorius is an award-winning educator and researcher in the fields of academic language, literacy, research skills, and research methodologies. 

Lecturers play a pivotal role in shaping the learning of their students. In a metric-focused university environment, this learning necessitates the assessment of students’ learning throughout their educational journey. Assessing assignments not only gauges the understanding of the subject matter but also evaluates the development of critical academic skills. These skills, such as research, analysis, and effective communication, are integral components of a well-rounded higher education.

Assessing transferable skills

The skills assessed must align with what is taught within the unit. When students perceive a direct connection between what is taught and what is assessed, their engagement and comprehension are heightened. Consequently, if we are going to assess students not only on their content knowledge but also their transferable skills, we need to provide them with the tools to succeed.

I believe that transferable skills enhance the applicability of students’ disciplinary knowledge. For years, I have worked to develop a suite of academic skills resources which are now embedded across the units within our Faculty. These resources include a suite of just-in-time online videos freely available on YouTube, as well as two written booklets (Doing Assignments and Writing Theses) that explicitly teach academic communication skills.

Over the years, I have also worked to improve the assignment rubrics within our Faculty to more accurately assess the skills that are taught within individual units. For example, I have worked with another staff member to develop templates for staff to provide feedback on academic language and literacy. We designed these templates to allow assessors to label specific mistakes for students and to provide students with referrals to appropriate support. Giving students specific labels for their errors helps them to see where they can improve. The referrals to appropriate resources and support help the student improve their skills, encouraging self-directed learning.

It is important to note that we usually recommend that these skills account for no more than 10% of the total grade for the assignment. This is because the main focus of the assessment should be the content – students should be able to clearly demonstrate an understanding and critical evaluation of the topic of the assignment. However, the students’ use of academic language and academic literacy can enhance the quality of their disciplinary content, or it can hinder the meaning of their ideas. As such, our templates allow for 5% to be attached to academic language (specifically, the elements listed in blue here) and 5% to academic literacy (the elements listed in purple here).

Assessing AI literacy

In the era of rapid technological advancement, the rise of generative artificial intelligence (AI) introduces a new dimension to education. As students are increasingly exposed to AI tools, it becomes imperative for educators to teach them how to use these tools effectively. As I have highlighted in another blog post, I firmly believe that it is our role as educators to teach students how to collaborate effectively with AI and evaluate the results obtained, a concept termed AI literacy. I see AI literacy as an essential transferable skill.

A key component of using AI ethically is acknowledging it effectively in written work. It is important to highlight, though, that if we are going to require students to demonstrate AI literacy, including the accurate acknowledgement of the use of AI tools, we need to teach it in our units and also assess it accurately. In my units, I teach students that an acknowledgement should include the name of the AI used, a description of how it was used (including the prompt used where appropriate), and an explanation of how the information was then adapted in the final version of the document. I also provide students with the example below so that they can see how an acknowledgement is used in practice.

I acknowledge that I used ChatGPT (OpenAI, https://chat.openai.com/) in this assignment to improve my written expression quality and generate summaries of the six articles I used in the annotated bibliography section. The summary prompt provided to the AI was “Write a 350 word abstract for this article. Include a summary of the topic of the article, the methodology used, the key findings, and the overall conclusion”. I adapted the summaries it produced to reflect my argument, style, and voice. I also adapted the summaries to better link with my topic under investigation. When I wanted the AI to help me improve my writing clarity, I pasted my written text and asked it to rewrite my work “in less words”, “in a more academic style”, or “using shorter sentences”. I also asked it to explain why it made the changes it did so that I could use this collaborative discussion as a learning process to improve my academic communication skills.

Clear guidelines within rubrics should also be established to evaluate the ethical and responsible use of AI, reinforcing the importance of acknowledging the role of these tools in academic work. Given my previous work developing rubric templates for staff, I have recently developed a template for the acknowledgement of AI use within assignments. In my template, this criterion falls within the “academic literacy” section of the rubric I mentioned earlier. I have included the rubric criteria below so that other educators can use it as needed. The grading scale is the one used in my university, but it can be easily adapted to other grading scales.

• High Distinction (80-100%): There was an excellent explanation about how generative AI software was used. This included, where appropriate, explicit details about the software used, the prompts provided to the AI, and explanations as to how the output of the generative AI was adapted for use within the assignment.  
• Distinction (70-79%): There was a clear explanation about how generative AI software was used. This included, where appropriate, sufficient detail about the software used, the prompts provided to the AI, and explanations as to how the output of the generative AI was adapted for use within the assignment. 
• Credit (60-69%): There was a reasonably clear explanation about how generative AI software was used. The explanation lacked sufficient details regarding one of the following: the software used, the prompts provided to the AI, and/or explanations as to how the output of the generative AI was adapted for use within the assignment.
• Pass (50-59%): There was some explanation about how generative AI software was used. The explanation lacked several of the following: the software used, the prompts provided to the AI and/or explanations as to how the output of the generative AI was adapted for use within the assignment.
• Fail (Below 50%): There was little or no explanation about how generative AI software was used.

Questions to ponder

The blog post outlines a rubric for assessing the acknowledgement and use of generative AI in student assignments. Considering the varying levels of detail and adaptation of AI-generated content required for different grades, what are your thoughts on the fairness and effectiveness of this approach?

How might this rubric evolve as generative AI technology becomes more advanced and commonplace in educational environments?