How People Can Create—and Destroy—Value with Generative AI: Research Insights

The Boston Consulting Group (BCG) authored, in collaboration with Harvard Business School, MIT Sloan School of Management, the Wharton School at the University of Pennsylvania, and the University of Warwick, a scientific experiment that finds that people mistrust generative AI in areas where it can contribute tremendous value and trust it too much where the technology isn’t competent.

Some of the key findings from the paper are summarized below:

• Mistrust and Overtrust: People mistrust generative AI in areas where it excels (creative product innovation) and trust it too much where it is less competent (problem solving).
• Performance Boost: Using GPT-4 for creative tasks improved performance by 40%, but for business problem-solving, it decreased performance by 23%
• Training Paradox: Training human users on GPT-4’s limitations didn’t mitigate negative effects and sometimes worsened performance.
• Creativity Risks: Extensive use of GPT-4 may reduce individual creativity and diversity of ideas.

These findings are obviously extremely interesting and immensely thought-provoking. It’s also worth noting that this study (and the associated research paper) date back to September 2023. Since then, LLMs and Generative AI have made significant improvements.

• What has your own and your organization’s experience been with respect to the balance between creativity and business problem solving skills of generative AI?
• Have you experienced the creative task performance boost that BCG observed?
• Have you noticed increased uniformity of outputs from your teams?
• Do you believe that Generative AI will be (or already is) able to solve business problems?

You can find the full article here, and the academic version here.