Wisconsin School of Business students had a hand in what is the first known crowdsourced accounting study to ever take place—and outdid ChatGPT in the process.
Summary data from WSB Distinguished Teaching Professor Ann O’Brien’s (PhD ’17) Accounting Systems course last fall along with data from 185 other institutions across 14 countries was used for the experiment, which pitted the artificial intelligence chatbot, ChatGPT, against students in answering 28,085 accounting-based questions on topics ranging from audit and finance to tax and managerial content.
Judged on the number of correct responses to questions, results released as of January 2023 indicated that humans (students) outperformed ChatGPT by answering 76.7% of the questions correctly to the chatbot’s 56.5%. Not only did students overall exceed ChatGPT, but Wisconsin students did as well.
“The paper is noteworthy in two ways,” says O’Brien. “One, the way in which data was collected, compiled, and crowdsourced. And two, the content—ChatGPT. It’s one of the first research papers that looks at how ChatGPT, the artificial intelligence chatbot, performs compared to accounting students on exam questions.”
The study was just one of the ways O’Brien prepares her students to learn more about AI and other key tools while in school. Her Spring 2023 course, Foundation in Accounting Analytics, focused on emerging technologies and gives students the opportunity to practice applying them as future accounting professionals. Some of the analytical tools and technologies covered during the course include SAP Analytics, Tableau, Alteryx, Power BI, UiPath, and XBRL.
“In the class, students adopt a professional approach as accountants who are dealing with accounting analytics,” says O’Brien. “They are exploring different analytic tools and topics, everything from data management and cleaning ETL [a way of cleaning data short for ‘exact, transform, and load’] to visualizations, business modeling, automation, and financial statement analysis.”
She says students also have the freedom to “dive deeper and explore their interests” during the course and choose a technology—like ChatGPT—to then learn and teach to each other.
“It was fun to work on an innovative research project; and hopefully, my students are also inspired by opportunities to engage with the transformative impact of digital technologies,” O’Brien says.
O’Brien has invited notable speakers, many of whom are alumni, working in industry to share their expertise with the class. Professionals from PwC and Johnson Controls have visited to talk about how they use Alteryx, a data cleaning tool, and Power BI, a data modeling technology. WSB accounting alum John Paetsch (BBA ’17, MA ’18), formerly a business intelligence manager at Epic who now works at Google as a finance data and analytics engineer, took students through the analytics exercise he does with his team. Mike Kreemer (BBA ’06, MA ’07), senior director in the sales program office with Workiva, discussed XBRL and the democratization of data with the class.