Below is a summary of the article: Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. MIT Working Paper.
I’ve created a collection of summaries to help me keep track of the academic research I’m reading about Artificial Intelligence in Higher Education. For more summaries, visit: broneager.com/AI-HE-Research
Summary
The paper examines the productivity effects of a generative artificial intelligence technology, specifically the assistive chatbot ChatGPT, in mid-level professional writing tasks.
The authors conducted a preregistered online experiment where they assigned occupation-specific, incentivised writing tasks to college-educated professionals and randomly exposed half of them to ChatGPT.
The sample in the study consisted of 444 college-educated professionals. Descriptive statistics about the sample, including annual salary and occupation, are provided in Table 1 (p. 14).
Prior to the study, 70% of participants were aware of ChatGPT, with 30% having used it before.
They found that ChatGPT raises average productivity, decreases the time taken by 0.8 SDs, and increases output quality by 0.4 SDs. It also benefits low-ability workers more, compresses productivity distribution, and restructures tasks towards idea generation and editing.
It appears that the intervention (using ChatGPT) amplifies existing abilities.
The implications of this study are significant for both industry and education. In industry, generative AI technologies like ChatGPT can be used to improve productivity and quality in writing tasks.
…a potent generative writing tool like ChatGPT might entirely replace certain kinds of writers, such as grant writers or marketers, by letting companies directly automate the creation of grant applications and press releases with minimal human oversight. This might not increase the quality of the resulting written output, but would let companies save on wage costs by eliminating human labor.”
Noy and Zhang (2023)
In education, one implication is that AI technologies could be used to support students in their writing assignments and help them develop their skills in idea generation and editing.
Furthermore, this study highlights the need for educators to incorporate AI technologies into their teaching practices to prepare students for the changing job market. Students need to learn how to work alongside AI technologies effectively and develop skills that complement these technologies rather than compete with them.
Overall, this study provides insights into the potential benefits of generative AI technologies like ChatGPT in improving productivity and quality in writing tasks. It also highlights the importance of incorporating AI technologies into education to prepare students for the changing job market.
Thanks to the authors for publishing the study!
You can access the paper online.