A six-month experiment by researchers from UC Berkeley, Harvard, and Columbia University tested how a GPT-4-powered AI advisor influenced small business owners in Kenya.
The most capable entrepreneurs increased their profits by 15 percent after using the tool. They treated the model’s suggestions as starting points rather than ready-made instructions and adapted them to real-world constraints, such as purchasing generators to handle frequent power outages or expanding their service offerings.
In contrast, less experienced participants lost approximately 10 percent of their income. They frequently implemented the AI’s generic recommendations without scrutiny, for example by lowering prices in ways that reduced overall revenue despite appearing logical in the chat interface.
The study concludes that GPT-4 functions less as a business consultant and more as a highly confident generator of losses when users forgo critical evaluation. The researchers emphasize that the problem stems primarily from the human tendency to stop thinking critically rather than from any flaw in the underlying technology.
Supporting evidence comes from an earlier Dickinson College experiment in which 97 percent of participants copied a clearly wrong answer generated by ChatGPT on a straightforward task, while the control group without AI performed better. A simple prompt reminding users to verify the output doubled accuracy rates.
Analysis of 1.4 million work sessions at KPMG produced similar findings: roughly 95 percent of users request a draft or summary and accept the first result, whereas only 5 percent treat the model as a collaborative reasoning partner by supplying context, guiding the chain of thought, and questioning outputs.
For organizations, these patterns mean that simply providing access to generative AI tools says little about expected productivity gains. Universal guidelines can even prove counterproductive because different tasks require different degrees of human oversight. The technology can accelerate strong performers while convincingly leading weaker users in the wrong direction.
