Teachers worried about students turning in essays written by a popular artificial intelligence chatbot now have a new tool of their own.
Edward Tian, a 22-year-old senior at Princeton University, has built an app to detect whether text is written by ChatGPT, the viral chatbot that’s sparked fears over its potential for unethical uses in academia.
Edward Tian, a 22-year-old computer science student at Princeton, created an app that detects essays written by the impressive AI-powered language model known as ChatGPT.
Tian, a computer science major who is minoring in journalism, spent part of his winter break creating GPTZero, which he said can “quickly and efficiently” decipher whether a human or ChatGPT authored an essay.
His motivation to create the bot was to fight what he sees as an increase in AI plagiarism. Since the release of ChatGPT in late November, there have been reports of students using the breakthrough language model to pass off AI-written assignments as their own.
“there’s so much chatgpt hype going around. is this and that written by AI? we as humans deserve to know!” Tian wrote in a tweet introducing GPTZero.
Tian said many teachers have reached out to him after he released his bot online on Jan. 2, telling him about the positive results they’ve seen from testing it.
More than 30,000 people had tried out GPTZero within a week of its launch. It was so popular that the app crashed. Streamlit, the free platform that hosts GPTZero, has since stepped in to support Tian with more memory and resources to handle the web traffic.
To determine whether an excerpt is written by a bot, GPTZero uses two indicators: “perplexity” and “burstiness.” Perplexity measures the complexity of text; if GPTZero is perplexed by the text, then it has a high complexity and it’s more likely to be human-written. However, if the text is more familiar to the bot — because it’s been trained on such data — then it will have low complexity and therefore is more likely to be AI-generated.
Separately, burstiness compares the variations of sentences. Humans tend to write with greater burstiness, for example, with some longer or complex sentences alongside shorter ones. AI sentences tend to be more uniform.
In a demonstration video, Tian compared the app’s analysis of a story in The New Yorker and a LinkedIn post written by ChatGPT. It successfully distinguished writing by a human versus AI.
Tian acknowledged that his bot isn’t foolproof, as some users have reported when putting it to the test. He said he’s still working to improve the model’s accuracy.
But by designing an app that sheds some light on what separates human from AI, the tool helps work toward a core mission for Tian: bringing transparency to AI.
“For so long, AI has been a black box where we really don’t know what’s going on inside,” he said. “And with GPTZero, I wanted to start pushing back and fighting against that.”
Culled from MyEngineers