A new randomized controlled trial by Harvard and Stanford economists finds that ChatGPT can substantially boost productivity for professionals completing everyday writing tasks.

Participants were randomly assigned realistic work simulations with or without access to ChatGPT.

Those with access to the AI tool completed assignments 40% faster while producing 18% higher quality output as judged by expert evaluators.

The study adds nuance to debates on how AI will impact jobs, finding ChatGPT more likely to augment rather than replace human professionals.

ChatGPT Productivity Study

The research by economists Shoji Noy of Harvard University and Zixuan Zhang of Stanford University assigned over 450 college-educated professionals realistic writing assignments resembling their actual work.

Participants included marketers, consultants, data analysts, and other occupations producing written documents regularly.

The study reveals:

“Workers exposed to ChatGPT during the experiment were two times as likely to report using it in their real job two weeks after the experiment and 1.6 times as likely two months after.”

ChatGPT Reduces Skills Gap

The study found that ChatGPT reduced inequality between higher-skilled and lower-skilled professionals.

Participants who received lower scores on their first writing assignment improved more when given access to ChatGPT on their second task than the control group.

The study reads:

“In the treatment group, initial inequalities were more than half-erased by the treatment. This reduction in inequality was driven by the fact that participants who scored lower on the first task benefited more from ChatGPT access.”

ChatGPT More Likely To Help Humans Than Replace Them

The authors say their results suggest that AI tools like ChatGPT are more likely to augment human professionals rather than replace them.

However, they caution that their brief experiment examined a limited range of occupations and incentive structures.

“Relative to our experimental findings, the direct productivity effects of ChatGPT in the real economy will be somewhat lower, and the technology will be more strongly complementary to human workers.”

The researchers call for further studies on how productivity gains from AI translate into wages, employment, and demand changes across the broader economy.

Their experiment indicates that advanced AI tools will profoundly impact businesses and labor markets that are challenging to predict.

More Details On The Study Methodology

The study was a randomized controlled trial, considered the gold standard for evaluating interventions. Participants were randomly assigned to either have access to ChatGPT or not.

The writing tasks were designed based on actual activities professionals do on the job, such as writing reports, press releases, emails, etc. Participants rated the tasks as realistic.

Output quality was evaluated by experienced professionals blind to the experimental condition. Participants were incentivized for high-quality work.

Participants were incentivized with bonus payments based on the quality of their writing as judged by expert evaluators blind to the study conditions. On average, participants earned $17 per hour.

Conclusion

This randomized trial reveals that ChatGPT can boost productivity. However, the study examined a limited scope of occupations and incentives.

As advanced AI diffuses across the economy, it’s important to continue investigating its impacts on wages, employment, inequality, and more.

While the technology may augment professionals, broader economic effects remain complex and challenging to predict.

Featured Image: Vitor Miranda/Shutterstock