Using AI for Presentation Feedback
Practical Tips and Reflections
AI tools give students the ability to get feedback on their presentations with potential for significant improvement in the final version. For this to be useful the student needs to be organised and have their presentation ready ahead of time, they also need to know the mechanics of how to do this, which is why I have provided this advice to them.
The specific problem that we are trying to fix is that students will find it hard to get any feedback on a presentation they are going to give. Academics will rarely provide it in a very comprehensive way, and whilst students may have peers or family members that can help, this will often not be the case, and the feedback may not be of great quality. Compared to this, whilst AI feedback may not be perfect it is likely much better than what students would have without it.
For my own course I have told students that I expect them to use this, and as a consequence I have higher standards for the final presentation, which is also connected to the potential for AI to help with planning, with production of visuals and as a general tutor.
Because I want students to be competent in using AI to help them, and don’t want them to hold back in its use, I have added two elements to the mark scheme for the presentations on my course.
1. They need to reflect on the tools they have used to create their presentations, including, but not limited to their use of AI.
2. I will be asking questions at the end of their presentations and they are marked directly on their ability to respond. They therefore need to know what they are talking about and understand the contents of their presentation.
I feel that both of these are necessary to give both the academic and the students the confidence to fully embrace the use of AI. For the academic you know you have a mechanism to tell whether a student has tried to take shortcuts in their actual understanding of the material and are encouraging them to think about their use of tools. For the student they know they need to be able to respond but are also being encouraged to use these tools in a way they will hopefully embrace so they can produce better work and get a more stimulating learning experience. They can also be encouraged that the marking will be fair in a way they might not if they are told not to use AI for certain things, but know that some in the group probably are using it surreptitiously to get an advantage that may not be detected.
I have included some examples of the sorts of chats you might have about a presentation on my website, some reflection on this might be useful.....
First in terms of practicalities, there is the option to simply present to the AI, which directly transcribes your words. Or there is the option to record on a different platform that provides a text transcription, and feed that transcription to the AI for feedback. In both cases length can be an issue, though with student presentations of 10-15 minutes this should normally not be a significant problem, with that amount of text normally fitting in the context window for most common AI tools (ChatGPT, Gemini, Claude, Bing for example). It is also possible to get a good service for free, though as of winter 2023-24, a ChatGPT premium subscription will get the best results.
Comparing the different outputs, I find that all of them provide something useful, but there are notable differences. In my experiments I deliberately gave an incorrect explanation about supply and demand in economics. I would ideally like the AI to be very clear that it was wrong, and to help me understand why it was wrong and what a correct answer would look like. LLMs are trained to be helpful and friendly but this is not what I really need here, I don’t need it to be horrible, but I need it to be direct.
In the experiment I found that Bing/Copilot is straight to the point and provides references to follow-up on. In comparison Bard (now Gemini) was less clear that my explanation was wrong, but did provide a useful correct explanation, with ChatGPT 3.5 being similar. This performance boost for Bing/Copilot is likely the result of using the GPT4 model in the underlying analysis, which has significantly better performance across a range of metrics. This is confirmed by using the same prompts in GPT4, where the AI is much clearer that I am wrong, and the explanation for why I am wrong is more detailed. At
Using the text copying method I found that Bard1 did not perform that well, you had to push it to be critical. Claude on the other hand was very direct in its criticism in a way that I found helpful, it is less interested in being ‘nice’ than other models used in this experiment, and in getting feedback, this is what you should want. Combined with the ability to directly link files, this makes Claude a strong option for this process.
In summary, ChatGPT premium is the best option for those that have access. If not, then using the microphone option in Bing will be a good option, or using the transcript copying method into Claude. As with any use of AI, some more detail in the prompts and in follow-up questions in the conversation will yield better results, and it is important to evaluate any feedback in combination with your own knowledge about what makes good presentations and of the subject matter,
I attach a fairly high probability (p > 0.5) that any student using this process will be able to produce a better final presentation. Having now had a round of student presentations, several students confirmed that they did use this procedure and had felt that it helped them. Something more robust to compare performance between those using AI feedback and those who don’t could be a potential research project for the future.
An important potential drawback is for students to focus on the feedback from the AI to the detriment of wider thinking about the issue. It should be emphasised that the AI feedback will likely raise some important points but that they should continue to reflect widely on how to complete their presentation to the best possible standard using a range of tools and knowledge.
See my website for full details including slides (that can be adapted for your own institution) and example chats.
This experiment was done before Bard became Gemini. I have yet to try this out with the better version of Gemini, so we will see if it produces an improvement.