Two professors who say they caught students cheating on essays with ChatGPT explain why AI plagiarism can be hard to prove
- Two philosopher professors said they caught their students submitting essays written by ChatGPT.
- They said certain red flags alerted them to the use of AI.
A few weeks after the launch of the AI chatbot ChatGPT, Darren Hick, a philosophy professor at Furman University, said he caught a student turning in an AI-generated essay.
Hick said he grew suspicious when the student turned in an on-topic essay that included some well-written misinformation.
After running it through Open AI's ChatGPT detector, the results said it was 99% likely the essay had been AI-generated.
Antony Aumann, a religious studies and philosophy professor at Northern Michigan University, told Insider he had caught two students submitting essays written by ChatGPT.
After the writing style set off alarm bells, Aumann submitted them back to the chatbot asking how likely it was that they were written by the program. When the chatbot said it was 99% sure the essays were written by ChatGPT, he forwarded the results to the students.
Both Hick and Aumann said they confronted their students, all of whom eventually confessed to the infraction. Hick's student failed the class and Aumann had his students rewrite the essays from scratch.
'It was really well-written wrong'
There were certain red flags in the essays that alerted the professors to the use of AI. Hick said the essay he found referenced several facts not mentioned in class, and made one nonsensical claim.
"Word by word it was a well-written essay," he said, but on closer inspection, one claim about the prolific philosopher, David Hume "made no sense" and was "just flatly wrong."
"Really well-written wrong was the biggest red flag," he said.
For Aumann, the chatbot just wrote too perfectly. "I think the chat writes better than 95% of my students could ever," he said.
"All of a sudden you have someone who does not demonstrate the ability to think or write at that level, writing something that follows all the requirements perfectly with sophisticated grammar and complicated thoughts that are directly related to the prompt for the essay," he said.
Christopher Bartel, a professor of philosophy at Appalachian State University, said that while the grammar in AI-generated essays is almost perfect, the substance tends to lack detail.
He said: "They are really fluffy. There's no context, there's no depth or insight."
If students don't confess to using AI for essays, it can leave academics in a tough spot.
Bartel said that some institutions' rules haven't evolved to combat this kind of cheating. If a student decided to dig their heels in and deny the use of AI, it can be difficult to prove.
Bartel said the AI detectors on offer were "good but not perfect."
"They give a statistical analysis of how likely the text is to be AI-generated, so that leaves us in a difficult position if our policies are designed so that we have to have definitive and demonstrable proof that the essay is a fake," he said. "If it comes back with a 95% likelihood that the essay is AI generated, there's still a 5% chance that it wasn't."
In Hick's case, although the detection site said it was "99% certain" the essay had been generated by an AI, he said it wasn't enough for him without a confession.
"The confession was important because everything else looks like circumstantial evidence," he said. "With AI-generated content, there is no material evidence, and material evidence has a lot more weight to it than circumstantial evidence."
Aumann said although he thought the analysis by the chatbot would be good enough proof for disciplinary action, AI plagiarism was still a new challenge for colleges.
He said: "Unlike plagiarism cases of old where you can just say, 'hey, here's the paragraph from Wikipedia.' There is no knockdown proof that you can provide other than the chat says that's the statistical likelihood."
- OpenAI CEO Sam Altman is privately reassuring developers using the company's tech that it won't compete with them beyond ChatGPT
- Media mogul Martha Stewart isn't a fan of remote work: 'You can't possibly get everything done working three days a week in the office and two days remotely'
- My twin and I were in separate classrooms in school. Our joint math lesson made me realize how silly that was.
- Dalal Street welcomes RBI MPC holding interest rate as but wary of inflation focus
- Best affordable 2-in-1 convertible laptops in India for 2023
- RBI MPC holds interest rate as expected but stance maintained at ‘withdrawal of accommodation’
- Salesforce unveils disruptive AI-backed solutions that can create personalised content in a single click
- Sacred Games to Scam 1992, here are India's most popular web series as per IMDb